vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #410
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.A0LjLe6tnb', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15(finished: start all VMs, in 66.64 seconds)16gitlab # Filesystem UUID: 62c0504a-be13-435c-bc41-e55d2f03838317gitlab: waiting for unit setup-gitlab.service18gitlab # Superblock backups stored on blocks:19gitlab: waiting for the VM to finish booting20gitlab # 32768, 98304, 163840, 22937621gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qs4h4v1di2h57w2wlrlnqjz9gwzips85-nixos-system-gitlab-test/init regInfo=/nix/store/8d0xn4nry4msl430sngzbqpzlbg4g3pb-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/8d0xn4nry4msl430sngzbqpzlbg4g3pb-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000028] arm-pv: using stolen time PV108gitlab # [ 0.000376] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000553] Console: colour dummy device 80x25110gitlab # [ 0.000559] printk: legacy console [tty0] enabled111gitlab # [ 0.000743] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000749] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000826] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000992] landlock: Up and running.115gitlab # [ 0.000995] Yama: becoming mindful.116gitlab # [ 0.001397] LSM support for eBPF active117gitlab # [ 0.001591] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001703] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002927] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003605] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003609] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003769] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004835] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.004929] EFI services will not be available.125gitlab # [ 0.005095] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005605] Detected PIPT I-cache on CPU1127gitlab # [ 0.005665] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005728] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.005789] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.006749] Detected PIPT I-cache on CPU2131gitlab # [ 0.006876] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.006948] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007020] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.007824] Detected PIPT I-cache on CPU3135gitlab # [ 0.007893] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.007965] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.008036] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.008336] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.008364] SMP: Total of 4 processors activated.140gitlab # [ 0.008367] CPU: All CPU(s) started at EL1141gitlab # [ 0.008375] CPU features: detected: Branch Target Identification142gitlab # [ 0.008378] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.008381] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.008385] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.008388] CPU features: detected: Common not Private translations146gitlab # [ 0.008391] CPU features: detected: CRC32 instructions147gitlab # [ 0.008393] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.008396] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.008399] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.008402] CPU features: detected: E0PD151gitlab # [ 0.008404] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.008407] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.008410] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.008412] CPU features: detected: Fine Grained Traps155gitlab # [ 0.008415] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.008419] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.008421] CPU features: detected: LSE atomic instructions158gitlab # [ 0.008424] CPU features: detected: Privileged Access Never159gitlab # [ 0.008426] CPU features: detected: PMUv3160gitlab # [ 0.008428] CPU features: detected: RAS Extension Support161gitlab # [ 0.008431] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.008433] CPU features: detected: Random Number Generator163gitlab # [ 0.008435] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.008438] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.008440] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.008444] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.008538] alternatives: applying system-wide alternatives168gitlab # [ 0.011900] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.012329] Memory: 5963808K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.013401] devtmpfs: initialized171gitlab # [ 0.017609] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.017683] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.017893] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.017898] 0 pages in range for non-PLT usage175gitlab # [ 0.017899] 508288 pages in range for PLT usage176gitlab # [ 0.018056] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.019196] DMI not present or invalid.178gitlab # [ 0.022857] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.025523] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.025778] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.027713] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.027735] audit: initializing netlink subsys (disabled)183gitlab # [ 0.028210] audit: type=2000 audit(0.028:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.028725] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.028727] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.028731] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.028734] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.028737] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.028804] cpuidle: using governor ladder190gitlab # [ 0.028846] cpuidle: using governor menu191gitlab # [ 0.029033] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.029159] ASID allocator initialised with 65536 entries193gitlab # [ 0.030301] Serial: AMBA PL011 UART driver194gitlab # [ 0.035626] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.035896] printk: console [ttyAMA0] enabled196gitlab # [ 0.041783] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.041793] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.041798] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.041802] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.041807] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.041811] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.041816] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.041820] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.049580] fbcon: Taking over console205gitlab # [ 0.049593] ACPI: Interpreter disabled.206gitlab # [ 0.050960] iommu: Default domain type: Translated207gitlab # [ 0.050969] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.061057] SCSI subsystem initialized209gitlab # [ 0.061533] usbcore: registered new interface driver usbfs210gitlab # [ 0.061570] usbcore: registered new interface driver hub211gitlab # [ 0.061613] usbcore: registered new device driver usb212gitlab # [ 0.061911] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.061915] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.061923] PTP clock support registered215gitlab # [ 0.061960] EDAC MC: Ver: 3.0.0216gitlab # [ 0.062329] scmi_core: SCMI protocol bus registered217gitlab # [ 0.063244] FPGA manager framework218gitlab # [ 0.064517] vgaarb: loaded219gitlab # [ 0.065036] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.066027] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.066066] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.066635] netfs: FS-Cache loaded223gitlab # [ 0.066857] pnp: PnP ACPI: disabled224gitlab # [ 0.073237] NET: Registered PF_INET protocol family225gitlab # [ 0.074768] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.115546] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.115612] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.115668] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.116042] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.116872] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.116996] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.117115] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.117592] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.117814] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.117843] NET: Registered PF_XDP protocol family236gitlab # [ 0.117861] PCI: CLS 0 bytes, default 64237gitlab # [ 0.118080] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.121384] kvm [1]: HYP mode not available239gitlab # [ 0.176699] Initialise system trusted keyrings240gitlab # [ 0.176864] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.177511] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.177626] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.190372] Key type asymmetric registered244gitlab # [ 0.190376] Asymmetric key parser 'x509' registered245gitlab # [ 0.190448] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.190671] io scheduler mq-deadline registered247gitlab # [ 0.190676] io scheduler kyber registered248gitlab # [ 0.202351] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.205269] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.205819] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.205834] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.205845] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.205851] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.205873] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.205898] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.205989] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.205998] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.206003] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.206006] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.206010] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.206103] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.206706] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.206956] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.206971] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.207015] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.207030] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.207633] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.207888] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.207901] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.207933] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.208553] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.208792] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.208806] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.208843] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.209465] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.209710] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.209725] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.209762] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.210387] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.210633] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.210647] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.210684] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.211286] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.211519] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.211532] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.211565] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.212140] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.212362] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.212374] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.212403] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.212416] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.213057] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.213307] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.213335] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.213901] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.214128] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.214156] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.214643] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.214889] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.215183] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.215506] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.215520] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.215554] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.216122] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.216349] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.216362] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.216395] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.217083] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.217091] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.217095] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.217151] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.217208] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.217264] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.217316] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.217375] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.217427] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.217474] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.217522] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.217576] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.217636] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.217683] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.217702] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.217727] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.217750] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.217773] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.217797] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.217817] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.217836] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.217856] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.217876] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.217896] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.217920] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.217938] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.217957] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.217980] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.218003] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.218027] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.218047] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.218066] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.218090] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.218109] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.218132] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.218138] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.218141] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.219644] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.236199] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.238635] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.240974] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.243328] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.245714] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.249902] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.252365] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.254682] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.257028] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.259126] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.263758] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.270517] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.272735] msm_serial: driver initialized359gitlab # [ 0.272884] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.272936] STM32 USART driver initialized361gitlab # [ 0.298182] loop: module loaded362gitlab # [ 0.298432] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.300713] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.302942] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.304907] virtio_blk virtio5: [vdb] 12394880 512-byte logical blocks (6.35 GB/5.91 GiB)366gitlab # [ 0.307559] megasas: 07.734.00.00-rc1367gitlab # [ 0.308611] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.310700] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.310719] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.312510] Using buffer write method371gitlab # [ 0.312548] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.314381] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.314396] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.316074] Using buffer write method375gitlab # [ 0.316093] Concatenating MTD devices:376gitlab # [ 0.316095] (0): "0.flash"377gitlab # [ 0.316097] (1): "0.flash"378gitlab # [ 0.316100] into device "0.flash"379gitlab # [ 0.582654] Freeing initrd memory: 27184K380gitlab # [ 0.592347] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.597340] thunder_xcv, ver 1.0382gitlab # [ 0.597410] thunder_bgx, ver 1.0383gitlab # [ 0.597440] nicpf, ver 1.0384gitlab # [ 0.598409] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.598416] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.598450] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.598459] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.598494] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.598499] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.598543] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.598549] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.598767] sky2: driver version 1.30393gitlab # [ 0.601550] usbcore: registered new interface driver usb-storage394gitlab # [ 0.601861] ehci-pci 0000:00:0a.0: EHCI Host Controller395gitlab # [ 0.601898] usbcore: registered new interface driver usbserial_generic396gitlab # [ 0.601919] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1397gitlab # [ 0.601931] usbserial: USB Serial support registered for generic398gitlab # [ 0.602513] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.602891] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.604509] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.604545] rtc-pl031 9010000.pl031: setting system clock to 2026-10-03T06:59:20 UTC (1791010760)402gitlab # [ 0.605250] i2c_dev: i2c /dev entries driver403gitlab # [ 0.610132] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.610139] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.610590] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.611255] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.613098] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00408gitlab # [ 0.614057] hub 1-0:1.0: USB hub found409gitlab # [ 0.614117] hid: raw HID events driver (C) Jiri Kosina410gitlab # [ 0.614184] hub 1-0:1.0: 6 ports detected411gitlab # [ 0.614870] usbcore: registered new interface driver usbhid412gitlab # [ 0.614877] usbhid: USB HID core driver413gitlab # [ 0.639446] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.642498] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.642753] NET: Registered PF_INET6 protocol family416gitlab # [ 0.644376] Segment Routing with IPv6417gitlab # [ 0.644414] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.644522] NET: Registered PF_PACKET protocol family419gitlab # [ 0.644755] 9pnet: Installing 9P2000 support420gitlab # [ 0.647293] Key type dns_resolver registered421gitlab # [ 0.657111] registered taskstats version 1422gitlab # [ 0.657416] Loading compiled-in X.509 certificates423gitlab # [ 0.671842] Demotion targets for Node 0: null424gitlab # [ 0.672065] Key type .fscrypt registered425gitlab # [ 0.672070] Key type fscrypt-provisioning registered426gitlab # [ 0.672242] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.672273] ima: Allocated hash algorithm: sha1428gitlab # [ 0.672360] ima: No architecture policies found429gitlab # [ 0.673470] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.696802] clk: Disabling unused clocks431gitlab # [ 0.696829] PM: genpd: Disabling unused power domains432gitlab # [ 0.701039] Freeing unused kernel memory: 4736K433gitlab # [ 0.701270] Run /init as init process434gitlab # [ 0.729158] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.869106] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.018772] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.063269] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.063305] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.063342] systemd[1]: Detected architecture arm64.440gitlab # [ 1.063349] systemd[1]: Running in initrd.441gitlab # [ 1.063775] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.063910] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.125416] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.245140] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.248728] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.311381] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.336012] systemd[1]: Created slice Slice /system/modprobe.448gitlab # [ 1.336260] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.449gitlab # [ 1.336295] systemd[1]: Expecting device /dev/disk/by-label/nix-store...450gitlab # [ 1.336310] systemd[1]: Expecting device /dev/disk/by-label/nixos...451gitlab # [ 1.336335] systemd[1]: Reached target Path Units.452gitlab # [ 1.336351] systemd[1]: Reached target Slice Units.453gitlab # [ 1.336370] systemd[1]: Reached target Swaps.454gitlab # [ 1.336389] systemd[1]: Reached target Timer Units.455gitlab # [ 1.336561] systemd[1]: Listening on D-Bus System Message Bus Socket.456gitlab # [ 1.336741] systemd[1]: Listening on Journal Socket (/dev/log).457gitlab # [ 1.336912] systemd[1]: Listening on Journal Sockets.458gitlab # [ 1.337198] systemd[1]: Listening on udev Control Socket.459gitlab # [ 1.337299] systemd[1]: Listening on udev Kernel Socket.460gitlab # [ 1.337320] systemd[1]: Reached target Socket Units.461gitlab # [ 1.338999] systemd[1]: Starting Create List of Static Device Nodes...462gitlab # [ 1.340734] systemd[1]: Starting Load Kernel Module 9pnet_virtio...463gitlab # [ 1.340803] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs464gitlab # [ 1.342542] systemd[1]: Mounting Kernel Configuration File System...465gitlab # [ 1.358040] systemd[1]: Starting Journal Service...466gitlab # [ 1.360531] systemd[1]: Starting Load Kernel Modules...467gitlab # [ 1.360674] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os468gitlab # [ 1.362748] systemd[1]: Starting Coldplug All udev Devices...469gitlab # [ 1.367465] systemd[1]: Finished Create List of Static Device Nodes.470gitlab # [ 1.369377] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.471gitlab # [ 1.369872] systemd[1]: Finished Load Kernel Module 9pnet_virtio.472gitlab # [ 1.370256] systemd[1]: Mounted Kernel Configuration File System.473gitlab # [ 1.378043] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...474gitlab # [ 1.394662] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.475gitlab # [ 1.395028] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev476gitlab # [ 1.396455] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0477gitlab # [ 1.396806] systemd-journald[96]: Collecting audit messages is disabled.478gitlab # [ 1.396831] [drm] features: -virgl +edid -resource_blob -host_visible479gitlab # [ 1.396838] [drm] features: -context_init480gitlab # [ 1.398207] [drm] number of scanouts: 1481gitlab # [ 1.398233] [drm] number of cap sets: 0482gitlab # [ 1.399602] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2483gitlab # [ 1.401372] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0484gitlab # [ 1.401802] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic485gitlab # [ 1.401815] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0486gitlab # [ 1.403506] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.487gitlab # [ 1.409246] systemd[1]: Starting Create Static Device Nodes in /dev...488gitlab # [ 1.409577] Console: switching to colour frame buffer device 160x50489gitlab # [ 1.413679] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device490gitlab # [ 1.437977] systemd[1]: Finished Load Kernel Modules.491gitlab # [ 1.442119] systemd[1]: Starting Apply Kernel Variables...492gitlab # [ 1.452691] systemd[1]: Finished Create Static Device Nodes in /dev.493gitlab # [ 1.453257] systemd[1]: Reached target Preparation for Local File Systems.494gitlab # [ 1.453441] systemd[1]: Reached target Local File Systems.495gitlab # [ 1.458720] systemd[1]: Starting Rule-based Manager for Device Events and Files...496gitlab # [ 1.460968] systemd[1]: Started Journal Service.497gitlab # [ 1.456376] systemd-modules-load[97]: Using 4 probe threads498gitlab # [ 1.461047] systemd-modules-load[97]: Module 'virtio_balloon' is built in499gitlab # [ 1.462456] systemd-modules-load[97]: Module 'virtio_console' is built in500gitlab # [ 1.463748] systemd-modules-load[97]: Module 'virtio_rng' is built in501gitlab # [ 1.465218] systemd-modules-load[97]: Inserted module 'dm_mod'502gitlab # [ 1.466425] systemd-modules-load[97]: Inserted module 'virtio_gpu'503gitlab # [ 1.467584] systemd[1]: Finished Apply Kernel Variables.504gitlab # [ 1.468746] systemd[1]: Starting Create System Files and Directories...505gitlab # [ 1.480407] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.493303] systemd-udevd[108]: Using default interface naming scheme 'v261'.507gitlab # [ 1.508860] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.530983] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.578142] systemd-vconsole-setup[131]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.581616] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.039606] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.041096] systemd[1]: Reached target System Initialization.513gitlab # [ 2.042420] systemd[1]: Reached target Basic System.514gitlab # [ 2.181375] systemd[1]: Found device /dev/disk/by-label/nixos.515gitlab # [ 2.187676] systemd[1]: Found device /dev/disk/by-label/nix-store.516gitlab # [ 2.191574] systemd[1]: Reached target Initrd Root Device.517gitlab # [ 2.193648] (udev-worker)[145]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.518gitlab # [ 2.196697] (udev-worker)[145]: Network interface NamePolicy= disabled on kernel command line.519gitlab # [ 2.198292] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...520gitlab # [ 2.206412] (udev-worker)[143]: Network interface NamePolicy= disabled on kernel command line.521gitlab # [ 2.231720] systemd-fsck[153]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.234651] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.361267] systemd[1]: Mounting /sysroot...524gitlab # [ 2.401696] EXT4-fs (vda): mounted filesystem 62c0504a-be13-435c-bc41-e55d2f038383 r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.400672] systemd[1]: Mounted /sysroot.526gitlab # [ 2.401544] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.405097] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.408703] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.411420] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.425377] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.441148] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.459197] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.472249] systemd-sysroot-fstab-check[167]: /sysroot should be mounted in the initrd, will request daemon-reload.534gitlab # [ 2.484041] systemd[1]: Mounted /sysroot/nix/.rw-store.535gitlab # [ 2.486100] systemd[1]: Mounted /sysroot/run.536gitlab # [ 2.488790] systemd[1]: Mounted /sysroot/tmp/shared.537gitlab # [ 2.504756] erofs (device vdb): mounted with root inode @ nid 128.538gitlab # [ 2.506671] systemd[1]: Mounted /sysroot/tmp/xchg.539gitlab # [ 2.507463] systemd[1]: Starting rw-sysroot-nix-store.service...540gitlab # [ 2.510658] systemd[1]: Mounted /sysroot/nix/.ro-store.541gitlab # [ 2.511696] systemd[1]: Reload requested from client PID 167 ('systemd-sysroot') (unit initrd-parse-etc.service)...542gitlab # [ 2.513405] systemd[1]: Reloading...543gitlab # [ 2.620681] systemd[1]: Reloading finished in 108 ms.544gitlab # [ 2.674688] systemd-sysroot-fstab-check[167]: Requesting initrd-fs.target/start/replace...545gitlab # [ 2.676656] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.546gitlab # [ 2.680168] systemd[1]: Finished rw-sysroot-nix-store.service.547gitlab # [ 2.681137] systemd-sysroot-fstab-check[167]: Requesting swap.target/start/replace...548gitlab # [ 2.684214] systemd[1]: Starting rw-sysroot-nix-store.service...549gitlab # [ 2.687345] systemd[1]: initrd-parse-etc.service: Deactivated successfully.550gitlab # [ 2.690833] systemd[1]: Finished Mountpoints Configured in the Real Root.551gitlab # [ 2.692957] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.552gitlab # [ 2.701922] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.553gitlab # [ 2.703413] systemd[1]: Finished rw-sysroot-nix-store.service.554gitlab # [ 2.809391] (udev-worker)[141]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory555gitlab # [ 2.812837] (udev-worker)[141]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory556gitlab # [ 2.820381] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.557gitlab # [ 2.821942] systemd[1]: Stopped Virtual Console Setup.558gitlab # [ 2.822765] systemd[1]: Stopping Virtual Console Setup...559gitlab # [ 2.823726] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 2.836905] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.561gitlab # [ 2.838207] systemd[1]: Stopped Virtual Console Setup.562gitlab # [ 2.839038] systemd[1]: Starting Virtual Console Setup...563gitlab # [ 2.863480] systemd-vconsole-setup[197]: Configuration of first virtual console was skipped, ignoring remaining ones.564gitlab # [ 2.865832] systemd[1]: Finished Virtual Console Setup.565gitlab # [ 3.363535] systemd[1]: Mounting /sysroot/nix/store...566gitlab # [ 3.419093] systemd[1]: Mounted /sysroot/nix/store.567gitlab # [ 3.420643] systemd[1]: Reached target Initrd File Systems.568gitlab # [ 3.422019] systemd[1]: Starting Find NixOS closure...569gitlab # [ 3.423264] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...570gitlab # [ 3.451322] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.571gitlab # [ 3.453431] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.572gitlab # [ 3.463083] systemd[1]: Finished Find NixOS closure.573gitlab # [ 3.464497] systemd[1]: Reached target Initrd Default Target.574gitlab # [ 3.465877] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...575gitlab # [ 3.487632] systemd[1]: Stopped target Initrd Default Target.576gitlab # [ 3.489139] systemd[1]: Stopped target Basic System.577gitlab # [ 3.490340] systemd[1]: Stopped target Initrd Root Device.578gitlab # [ 3.491630] systemd[1]: Stopped target Path Units.579gitlab # [ 3.492888] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.580gitlab # [ 3.494631] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.581gitlab # [ 3.496478] systemd[1]: Stopped target Slice Units.582gitlab # [ 3.498200] systemd[1]: Stopped target Socket Units.583gitlab # [ 3.501287] systemd[1]: Stopped target System Initialization.584gitlab # [ 3.502864] systemd[1]: Stopped target Swaps.585gitlab # [ 3.503957] systemd[1]: Stopped target Timer Units.586gitlab # [ 3.506068] systemd[1]: dbus.socket: Deactivated successfully.587gitlab # [ 3.507612] systemd[1]: Closed D-Bus System Message Bus Socket.588gitlab # [ 3.509099] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.589gitlab # [ 3.510884] systemd[1]: Stopped Find NixOS closure.590gitlab # [ 3.512090] systemd[1]: Starting Load Kernel Module 9pnet_virtio...591gitlab # [ 3.513558] systemd[1]: Starting rw-sysroot-nix-store.service...592gitlab # [ 3.515113] systemd[1]: systemd-sysctl.service: Deactivated successfully.593gitlab # [ 3.517124] systemd[1]: Stopped Apply Kernel Variables.594gitlab # [ 3.518576] systemd[1]: systemd-modules-load.service: Deactivated successfully.595gitlab # [ 3.520327] systemd[1]: Stopped Load Kernel Modules.596gitlab # [ 3.521540] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.597gitlab # [ 3.523434] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.598gitlab # [ 3.525790] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.599gitlab # [ 3.527440] systemd[1]: Stopped Create System Files and Directories.600gitlab # [ 3.528915] systemd[1]: Stopped target Local File Systems.601gitlab # [ 3.530234] systemd[1]: Stopped target Preparation for Local File Systems.602gitlab # [ 3.531820] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.603gitlab # [ 3.533548] systemd[1]: Stopped Coldplug All udev Devices.604gitlab # [ 3.534856] systemd[1]: Stopping Rule-based Manager for Device Events and Files...605gitlab # [ 3.536558] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.606gitlab # [ 3.538216] systemd[1]: Stopped Virtual Console Setup.607gitlab # [ 3.539417] systemd[1]: initrd-cleanup.service: Deactivated successfully.608gitlab # [ 3.540999] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.609gitlab # [ 3.542473] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.610gitlab # [ 3.544077] systemd[1]: Finished rw-sysroot-nix-store.service.611gitlab # [ 3.545375] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.612gitlab # [ 3.547068] systemd[1]: Finished Load Kernel Module 9pnet_virtio.613gitlab # [ 3.548397] systemd[1]: systemd-udevd.service: Deactivated successfully.614gitlab # [ 3.549767] systemd[1]: Stopped Rule-based Manager for Device Events and Files.615gitlab # [ 3.551199] systemd[1]: systemd-udevd.service: Consumed 2.223s CPU time over 2.079s wall clock time, 32.3M memory peak.616gitlab # [ 3.553285] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.617gitlab # [ 3.554693] systemd[1]: Closed udev Control Socket.618gitlab # [ 3.555649] systemd[1]: Starting Cleanup udev Database...619gitlab # [ 3.556762] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.620gitlab # [ 3.558203] systemd[1]: Stopped Create Static Device Nodes in /dev.621gitlab # [ 3.559343] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.622gitlab # [ 3.560872] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.623gitlab # [ 3.562182] systemd[1]: kmod-static-nodes.service: Deactivated successfully.624gitlab # [ 3.563397] systemd[1]: Stopped Create List of Static Device Nodes.625gitlab # [ 3.595868] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.626gitlab # [ 3.597773] systemd[1]: Finished Cleanup udev Database.627gitlab # [ 3.599040] systemd[1]: Reached target Switch Root.628gitlab # [ 3.600259] systemd[1]: Starting NixOS Activation...629gitlab # [ 3.716907] initrd-nixos-activation-start[222]: booting system configuration /nix/store/qs4h4v1di2h57w2wlrlnqjz9gwzips85-nixos-system-gitlab-test630gitlab # [ 3.754490] initrd-nixos-activation-start[222]: running activation script...631gitlab # [ 4.002842] initrd-nixos-activation-start[245]: setting up /etc...632gitlab # [ 4.131140] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.633gitlab # [ 4.133151] systemd[1]: Finished NixOS Activation.634gitlab # [ 4.134508] systemd[1]: Starting Switch Root...635gitlab # [ 4.173265] systemd[1]: Switching root.636gitlab # [ 4.305878] systemd-journald[96]: Received SIGTERM from PID 1 (systemd).637gitlab # [ 4.903829] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)638gitlab # [ 4.903936] systemd[1]: Detected virtualization qemu.639gitlab # [ 4.904031] systemd[1]: Detected architecture arm64.640gitlab # [ 4.904210] systemd[1]: Detected first boot.641gitlab # [ 4.905437] systemd[1]: Initializing machine ID from random generator.642gitlab # [ 5.107825] systemd[1]: bpf-restrict-fs: LSM BPF program attached643gitlab # [ 5.176209] systemd[1]: Applying preset policy.644gitlab # [ 5.352986] systemd[1]: Populated /etc with preset unit settings.645gitlab # [ 5.538922] systemd[1]: initrd-switch-root.service: Deactivated successfully.646gitlab # [ 5.539451] systemd[1]: Stopped initrd-switch-root.service.647gitlab # [ 5.542390] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.648gitlab # [ 5.546247] systemd[1]: Created slice Slice /system/getty.649gitlab # [ 5.548533] systemd[1]: Created slice GitLab DevOps Platform Slice.650gitlab # [ 5.551115] systemd[1]: Created slice User and Session Slice.651gitlab # [ 5.551959] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.652gitlab # [ 5.552716] systemd[1]: Started Forward Password Requests to Wall Directory Watch.653gitlab # [ 5.553388] systemd[1]: Expecting device /dev/hvc0...654gitlab # [ 5.553962] systemd[1]: Expecting device /dev/ttyAMA0...655gitlab # [ 5.554541] systemd[1]: Reached target Local Encrypted Volumes.656gitlab # [ 5.555154] systemd[1]: Stopped target initrd-fs.target.657gitlab # [ 5.555726] systemd[1]: Stopped target initrd-root-fs.target.658gitlab # [ 5.556290] systemd[1]: Stopped target initrd-switch-root.target.659gitlab # [ 5.556872] systemd[1]: Reached target Virtual Machines and Containers.660gitlab # [ 5.557584] systemd[1]: Reached target Path Units.661gitlab # [ 5.558166] systemd[1]: Reached target Remote File Systems.662gitlab # [ 5.558733] systemd[1]: Reached target Slice Units.663gitlab # [ 5.559331] systemd[1]: Reached target Swaps.664gitlab # [ 5.561549] systemd[1]: Listening on Query the User Interactively for a Password.665gitlab # [ 5.563773] systemd[1]: Listening on Process Core Dump Socket.666gitlab # [ 5.565450] systemd[1]: Listening on Credential Encryption/Decryption.667gitlab # [ 5.567348] systemd[1]: Listening on Factory Reset Management.668gitlab # [ 5.568064] systemd[1]: Listening on Hostname Service Socket.669gitlab # [ 5.572889] systemd[1]: Starting Journal Log Access Socket...670gitlab # [ 5.574119] systemd[1]: Listening on Journal Audit Socket.671gitlab # [ 5.576428] systemd[1]: Listening on Console Output Muting Service Socket.672gitlab # [ 5.577316] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.673gitlab # [ 5.577936] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os674gitlab # [ 5.578490] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki675gitlab # [ 5.581757] systemd[1]: Listening on Disk Repartitioning Service Socket.676gitlab # [ 5.582532] systemd[1]: Listening on udev Control Socket.677gitlab # [ 5.583286] systemd[1]: Listening on udev Varlink Socket.678gitlab # [ 5.611553] systemd[1]: Mounting Huge Pages File System...679gitlab # [ 5.614407] systemd[1]: Mounting POSIX Message Queue File System...680gitlab # [ 5.617707] systemd[1]: Mounting Kernel Debug File System...681gitlab # [ 5.621392] systemd[1]: Mounting Kernel Trace File System...682gitlab # [ 5.627141] systemd[1]: Starting Create List of Static Device Nodes...683gitlab # [ 5.633210] systemd[1]: Starting Load Kernel Module 9pnet_virtio...684gitlab # [ 5.634449] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs685gitlab # [ 5.646674] systemd[1]: Mounting Kernel Configuration File System...686gitlab # [ 5.647381] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm687gitlab # [ 5.647999] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore688gitlab # [ 5.657118] systemd[1]: Starting Load Kernel Module fuse...689gitlab # [ 5.660135] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67690gitlab # [ 5.669751] systemd[1]: Starting Journal Service...691gitlab # [ 5.674810] systemd[1]: Starting Load Kernel Modules...692gitlab # [ 5.681230] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...693gitlab # [ 5.689217] systemd[1]: Starting Remount Root and Kernel File Systems...694gitlab # [ 5.689844] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os695gitlab # [ 5.696831] systemd[1]: Starting Coldplug All udev Devices...696gitlab # [ 5.696889] fuse: init (API version 7.45)697gitlab # [ 5.702977] systemd[1]: Listening on Journal Log Access Socket.698gitlab # [ 5.703748] systemd[1]: Mounted Huge Pages File System.699gitlab # [ 5.704568] systemd[1]: Mounted POSIX Message Queue File System.700gitlab # [ 5.705249] systemd[1]: Mounted Kernel Debug File System.701gitlab # [ 5.705928] systemd[1]: Mounted Kernel Trace File System.702gitlab # [ 5.706938] systemd[1]: Finished Create List of Static Device Nodes.703gitlab # [ 5.707490] systemd-journald[316]: Collecting audit messages is enabled.704gitlab # [ 5.707899] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.705gitlab # [ 5.708584] systemd[1]: Finished Load Kernel Module 9pnet_virtio.706gitlab # [ 5.709221] systemd[1]: Mounted Kernel Configuration File System.707gitlab # [ 5.709985] systemd[1]: modprobe@fuse.service: Deactivated successfully.708gitlab # [ 5.710796] systemd[1]: Finished Load Kernel Module fuse.709gitlab # [ 5.715076] systemd[1]: Mounting FUSE Control File System...710gitlab # [ 5.729073] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...711gitlab # [ 5.730094] systemd[1]: Finished Load Kernel Modules.712gitlab # [ 5.732754] systemd[1]: Starting Firewall...713gitlab # [ 5.737202] systemd[1]: Starting Apply Kernel Variables...714gitlab # [ 5.741116] EXT4-fs (vda): re-mounted 62c0504a-be13-435c-bc41-e55d2f038383.715gitlab # [ 5.737932] systemd[1]: Queued start job for default target Multi-User System.716gitlab # [ 5.740515] systemd[1]: Unnecessary job was removed for /dev/vdb.[ 5.746433] systemd[1]: Started Journal Service.717gitlab # 718gitlab # [ 5.746940] systemd[1]: systemd-journald.service: Deactivated successfully.719gitlab # [ 5.749416] systemd-modules-load[317]: Using 3 probe threads720gitlab # [ 5.751023] systemd-modules-load[317]: Module 'atkbd' is built in721gitlab # [ 5.760497] systemd-modules-load[317]: Module 'loop' is built in722gitlab # [ 5.761766] systemd-modules-load[317]: Inserted module 'tls'723gitlab # [ 5.762857] systemd[1]: Finished Remount Root and Kernel File Systems.724gitlab # [ 5.767064] systemd[1]: Listening on Disk Image Download Service Socket.725gitlab # [ 5.772496] systemd[1]: Starting Flush Journal to Persistent Storage...726gitlab # [ 5.773735] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore727gitlab # [ 5.782652] systemd-journald[316]: Received client request to flush runtime journal.728gitlab # [ 5.814569] systemd[1]: Starting Load/Save OS Random Seed...729gitlab # [ 5.815913] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os730gitlab # [ 5.817575] systemd[1]: Finished Apply Kernel Variables.731gitlab # [ 5.819204] systemd[1]: Mounted FUSE Control File System.732gitlab # [ 5.820363] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.733gitlab # [ 5.821665] systemd[1]: Starting Create Static Device Nodes in /dev...734gitlab # [ 5.823174] systemd-oomd[318]: No swap; memory pressure usage will be degraded735gitlab # [ 5.825071] systemd[1]: Finished Load/Save OS Random Seed.736gitlab # [ 5.826110] systemd[1]: Reached target First Boot Complete.737gitlab # [ 5.827457] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.738gitlab # [ 5.828639] systemd[1]: Finished Create Static Device Nodes in /dev.739gitlab # [ 5.829633] systemd[1]: Finished Flush Journal to Persistent Storage.740gitlab # [ 5.830626] systemd[1]: Reached target Preparation for Local File Systems.741gitlab # [ 5.831592] systemd[1]: Starting Rule-based Manager for Device Events and Files...742gitlab # [ 5.893568] systemd-udevd[356]: Using default interface naming scheme 'v261'.743gitlab # [ 5.933130] systemd[1]: Started Rule-based Manager for Device Events and Files.744gitlab # [ 6.122344] systemd[1]: Finished Firewall.745gitlab # [ 6.373894] systemd[1]: Finished Coldplug All udev Devices.746gitlab # [ 6.434051] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs747gitlab # [ 6.499485] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.748gitlab # [ 6.505301] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.749gitlab # [ 6.519067] (udev-worker)[408]: Network interface NamePolicy= disabled on kernel command line.750gitlab # [ 6.529919] (udev-worker)[406]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.751gitlab # [ 6.533196] (udev-worker)[406]: Network interface NamePolicy= disabled on kernel command line.752gitlab # [ 6.541605] systemd[1]: Mounting /run/wrappers...753gitlab # [ 6.551708] systemd[1]: Condition check resulted in Virtio network device being skipped.754gitlab # [ 6.553545] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore755gitlab # [ 6.555813] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67756gitlab # [ 6.559234] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore757gitlab # [ 6.564340] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os758gitlab # [ 6.566219] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os759gitlab # [ 6.575755] systemd[1]: Mounted /run/wrappers.760gitlab # [ 6.576804] systemd[1]: Reached target Local File Systems.761gitlab # [ 6.577927] systemd[1]: Listening on Boot Loader Control Service Socket.762gitlab # [ 6.581552] systemd[1]: Starting register-nix-paths.service...763gitlab # [ 6.582779] systemd[1]: Starting Create SUID/SGID Wrappers...764gitlab # [ 6.583861] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.765gitlab # [ 6.604520] systemd[1]: Starting Save Transient machine-id to Disk...766gitlab # [ 6.611748] mousedev: PS/2 mouse device common for all mice767gitlab # [ 6.616310] systemd[1]: Starting Create System Files and Directories...768gitlab # [ 6.665841] systemd[1]: Finished Create System Files and Directories.769gitlab # [ 6.669950] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.770gitlab # [ 6.672068] systemd[1]: Finished Save Transient machine-id to Disk.771gitlab # [ 6.678076] systemd[1]: Starting Rebuild Journal Catalog...772gitlab # [ 6.683500] systemd[1]: Starting Record System Boot/Shutdown in UTMP...773gitlab # [ 6.722431] systemd[1]: Finished Record System Boot/Shutdown in UTMP.774gitlab # [ 6.737202] systemd[1]: Finished Rebuild Journal Catalog.775gitlab # [ 6.740373] systemd[1]: Starting Update is Completed...776gitlab # [ 6.774126] systemd[1]: Finished Update is Completed.777gitlab # [ 6.805385] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.778gitlab # [ 6.808122] systemd[1]: Finished Create SUID/SGID Wrappers.779gitlab # [ 6.809343] systemd[1]: suid-sgid-wrappers.service: Consumed 157ms CPU time over 224ms wall clock time, 6.3M memory peak, 1.9M read from disk.780gitlab # [ 7.083889] systemd[1]: Finished register-nix-paths.service.[ 7.088846] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3781gitlab # 782gitlab # [ 7.090374] systemd[1]: Reached target System Initialization.783gitlab # [ 7.091430] systemd[1]: Started Discard unused filesystem blocks once a week.784gitlab # [ 7.092794] systemd[1]: Started logrotate.timer.785gitlab # [ 7.093542] systemd[1]: Started Daily Cleanup of Temporary Directories.786gitlab # [ 7.094599] systemd[1]: Reached target Timer Units.787gitlab # [ 7.097322] systemd[1]: Listening on D-Bus System Message Bus Socket.788gitlab # [ 7.098384] systemd[1]: Listening on Nix Daemon Socket.789gitlab # [ 7.123186] systemd[1]: Starting nixbot CI service socket...790gitlab # [ 7.124409] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.791gitlab # [ 7.125660] systemd[1]: Listening on nixbot CI service socket.792gitlab # [ 7.126527] systemd[1]: Reached target Socket Units.793gitlab # [ 7.152124] systemd[1]: Starting D-Bus System Message Bus...794gitlab # [ 7.214119] dbus-broker-launch[561]: Looking up NSS user entry for 'systemd-timesync'...795gitlab # [ 7.216381] dbus-broker-launch[561]: NSS returned no entry for 'systemd-timesync'796gitlab # [ 7.219649] dbus-broker-launch[561]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"797gitlab # [ 7.227671] systemd[1]: Started D-Bus System Message Bus.798gitlab # [ 7.236696] dbus-broker-launch[561]: Ready799gitlab # [ 7.243096] systemd[1]: Reached target Basic System.800gitlab # [ 7.250696] systemd[1]: Started backdoor.service.801gitlab # [ 7.251644] systemd[1]: Starting gitlab-config.service...802gitlab # [ 7.265659] systemd[1]: Starting Import lastlog data into lastlog2 database...803gitlab # [ 7.274384] systemd[1]: Starting Logrotate configuration check...804gitlab # [ 7.276648] systemd[1]: Starting Name Service Cache Daemon (nsncd)...805gitlab # connecting to host...806gitlab # [ 7.283188] systemd[1]: Starting Post-Boot Actions...807gitlab # [ 7.289456] systemd[1]: Started Reset console on configuration changes.808gitlab # [ 7.299366] logrotate[570]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.809gitlab: Guest shell says: b'Spawning backdoor root shell...\n'810gitlab # [ 7.301820] logrotate[570]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf811gitlab # [ 7.303341] logrotate[570]: note: 'monthly' overrides previously specified 'weekly'812gitlab # [ 7.304374] logrotate[570]: Reading state from file: /var/lib/logrotate.status813gitlab: connected to guest root shell814gitlab # [ 7.305268] logrotate[570]: state file /var/lib/logrotate.status does not exist815gitlab: (connecting took 7.75 seconds)816gitlab # [ 7.306285] logrotate[570]: Allocating hash table for state file, size 64 entries817gitlab: (finished: waiting for the VM to finish booting, in 7.75 seconds)818gitlab # [ 7.307228] logrotate[570]: Handling 4 logs819gitlab # [ 7.307796] logrotate[570]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed820gitlab # [ 7.309988] logrotate[570]: considering log /var/log/btmp821gitlab # [ 7.310708] logrotate[570]: Creating new state822gitlab # [ 7.311301] logrotate[570]: Now: 2026-10-03 06:59823gitlab # [ 7.311935] logrotate[570]: Last rotated at 2026-10-03 06:00824gitlab # [ 7.312807] logrotate[570]: log does not need rotating (log has already been rotated)825gitlab # [ 7.314074] logrotate[570]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed826gitlab # [ 7.316150] logrotate[570]: considering log /var/log/wtmp827gitlab # [ 7.316992] logrotate[570]: Creating new state828gitlab # [ 7.317606] logrotate[570]: Now: 2026-10-03 06:59829gitlab # [ 7.318273] logrotate[570]: Last rotated at 2026-10-03 06:00830gitlab # [ 7.319201] logrotate[570]: log does not need rotating (log has already been rotated)831gitlab # [ 7.320240] logrotate[570]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed832gitlab # [ 7.323415] logrotate[570]: switching euid from 0 to 165 and egid from 0 to 165 (pid 570)833gitlab # [ 7.324708] logrotate[570]: considering log /var/gitlab/state/log/*.log834gitlab # [ 7.326150] logrotate[570]: log /var/gitlab/state/log/*.log does not exist -- skipping835gitlab # [ 7.327330] logrotate[570]: Creating new state836gitlab # [ 7.328009] logrotate[570]: switching euid from 165 to 0 and egid from 165 to 0 (pid 570)837gitlab # [ 7.332099] logrotate[570]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed838gitlab # [ 7.334267] logrotate[570]: switching euid from 0 to 60 and egid from 0 to 60 (pid 570)839gitlab # [ 7.335419] logrotate[570]: considering log /var/log/nginx/*.log840gitlab # [ 7.339123] logrotate[570]: log /var/log/nginx/*.log does not exist -- skipping841gitlab # [ 7.341578] logrotate[570]: Creating new state842gitlab # [ 7.342376] logrotate[570]: not running postrotate script, since no logs were rotated843gitlab # [ 7.343629] logrotate[570]: switching euid from 60 to 0 and egid from 60 to 0 (pid 570)844gitlab # [ 7.345777] nsncd[573]: Oct 03 06:59:27.222 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"845gitlab # [ 7.348230] systemd[1]: Starting resolvconf update...846gitlab # [ 7.349841] systemd[1]: Starting Virtual Console Setup...847gitlab # [ 7.351075] systemd[1]: Started Name Service Cache Daemon (nsncd).848gitlab # [ 7.352326] systemd[1]: Finished Logrotate configuration check.849gitlab # [ 7.353171] systemd[1]: Finished Post-Boot Actions.850gitlab # [ 7.353945] systemd[1]: Reached target Host and Network Name Lookups.851gitlab # [ 7.354977] systemd[1]: Reached target User and Group Name Lookups.852gitlab # [ 7.356107] systemd[1]: Starting User Login Management...853gitlab # [ 7.357561] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.854gitlab # [ 7.358651] systemd[1]: Stopped Virtual Console Setup.855gitlab # [ 7.359402] systemd[1]: systemd-vconsole-setup.service: Consumed 26ms CPU time over 51ms wall clock time, 2.9M memory peak, 1M read from disk.856gitlab # [ 7.362023] systemd[1]: Finished Import lastlog data into lastlog2 database.857gitlab # [ 7.379544] systemd[1]: Starting Virtual Console Setup...858gitlab # [ 7.412246] systemd-logind[609]: New seat seat0.859gitlab # [ 7.417596] systemd-logind[609]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)860gitlab # [ 7.421632] systemd[1]: Stopped target Host and Network Name Lookups.861gitlab # [ 7.423930] systemd[1]: Stopping Host and Network Name Lookups...862gitlab # [ 7.425141] systemd[1]: Stopped target User and Group Name Lookups.863gitlab # [ 7.426145] systemd[1]: Stopping User and Group Name Lookups...864gitlab # [ 7.427737] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...865gitlab # [ 7.429245] systemd-logind[609]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)866gitlab # [ 7.430647] systemd-logind[609]: Watching system buttons on /dev/input/event0 (gpio-keys)867gitlab # [ 7.431925] systemd[1]: Started User Login Management.868gitlab # [ 7.432937] systemd-vconsole-setup[633]: Configuration of first virtual console was skipped, ignoring remaining ones.869gitlab # [ 7.434788] systemd[1]: nscd.service: Deactivated successfully.870gitlab # [ 7.435727] systemd[1]: Stopped Name Service Cache Daemon (nsncd).871gitlab # [ 7.436890] systemd[1]: nscd.service: Consumed 28ms CPU time over 145ms wall clock time, 7.8M memory peak, 5.8M read from disk.872gitlab # [ 7.438553] systemd[1]: Finished Virtual Console Setup.873gitlab # [ 7.439315] systemd[1]: Starting linger-users.service...874gitlab # [ 7.442458] systemd[1]: Starting Name Service Cache Daemon (nsncd)...875gitlab # [ 7.465326] systemd[1]: linger-users.service: Deactivated successfully.876gitlab # [ 7.466537] systemd[1]: Finished linger-users.service.877gitlab # [ 7.469955] systemd[1]: Finished resolvconf update.878gitlab # [ 7.471864] systemd[1]: Reached target Preparation for Network.879gitlab # [ 7.473956] nsncd[667]: Oct 03 06:59:27.373 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"880gitlab # [ 7.480095] systemd[1]: Starting DHCP Client...881gitlab # [ 7.481191] systemd[1]: Starting Address configuration of eth1...882gitlab # [ 7.487181] systemd[1]: Starting Extra networking commands....883gitlab # [ 7.488184] systemd[1]: Started Name Service Cache Daemon (nsncd).884gitlab # [ 7.489252] systemd[1]: Reached target Host and Network Name Lookups.885gitlab # [ 7.490260] systemd[1]: Reached target User and Group Name Lookups.886gitlab # [ 7.520899] network-addresses-eth1-start[694]: adding address 192.168.1.1/24... done887gitlab # [ 7.527783] network-addresses-eth1-start[694]: adding address 2001:db8:1::1/64... done888gitlab # [ 7.540530] systemd[1]: Finished Address configuration of eth1.889gitlab # [ 7.579581] dhcpcd[705]: dhcpcd-10.3.2 starting890gitlab # [ 7.583204] dhcpcd[758]: dev: loaded udev891gitlab # [ 7.583963] systemd[1]: Finished Extra networking commands..892gitlab # [ 7.587128] systemd[1]: Reached target Network.893gitlab # [ 7.589939] systemd[1]: Starting gitlab-workhorse.service...894gitlab # [ 7.596507] systemd[1]: Starting Nginx Web Server...895gitlab # [ 7.600551] systemd[1]: Starting PostgreSQL Server...896gitlab # [ 7.607044] 8021q: 802.1Q VLAN Support v1.8897gitlab # [ 7.607466] 8021q: adding VLAN 0 to HW filter on device eth1898gitlab # [ 7.606969] systemd[1]: Starting Redis Server - redis-gitlab...899gitlab # [ 7.612441] systemd[1]: Starting Permit User Sessions...900gitlab # [ 7.622643] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.901gitlab # [ 7.657770] cfg80211: Loading compiled-in X.509 certificates for regulatory database902gitlab # [ 7.667579] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'903gitlab # [ 7.668110] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'904gitlab # [ 7.671032] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2905gitlab # [ 7.671391] cfg80211: failed to load regulatory.db906gitlab # [ 7.676413] systemd[1]: Finished Permit User Sessions.907gitlab # [ 7.698946] systemd[1]: Started Getty on tty1.908gitlab # [ 7.699861] systemd[1]: Reached target Login Prompts.909gitlab # [ 7.722964] 8021q: adding VLAN 0 to HW filter on device eth0910gitlab # [ 7.720199] dhcpcd[758]: eth0: waiting for carrier911gitlab # [ 7.721107] dhcpcd[758]: eth0: carrier acquired912gitlab # [ 7.728426] dhcpcd[758]: DUID 00:01:00:01:32:53:64:4f:52:54:00:12:34:56913gitlab # [ 7.729570] dhcpcd[758]: eth0: IAID 00:12:34:56914gitlab # [ 7.730348] dhcpcd[758]: eth0: adding address fe80::5054:ff:fe12:3456915gitlab # [ 7.810304] nginx-pre-start[797]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok916gitlab # [ 7.812621] nginx-pre-start[797]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful917gitlab # [ 7.818470] systemd[1]: Started Nginx Web Server.918gitlab # [ 7.836523] redis-server[796]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.919gitlab # [ 7.839325] redis-server[796]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo920gitlab # [ 7.842325] postgresql-pre-start[794]: The files belonging to this database system will be owned by user "postgres".921gitlab # [ 7.844294] postgresql-pre-start[794]: This user must also own the server process.922gitlab # [ 7.845616] redis-server[796]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=796, just started923gitlab # [ 7.847293] systemd[1]: Started Redis Server - redis-gitlab.924gitlab # [ 7.848278] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8".925gitlab # [ 7.849617] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8".926gitlab # [ 7.851197] postgresql-pre-start[794]: The default text search configuration will be set to "english".927gitlab # [ 7.852743] postgresql-pre-start[794]: Data page checksums are enabled.928gitlab # [ 7.853724] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok929gitlab # [ 7.855237] postgresql-pre-start[794]: creating subdirectories ... ok930gitlab # [ 7.856895] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix931gitlab # [ 7.858475] redis-server[796]: Configuration loaded932gitlab # [ 7.859372] redis[796]: monotonic clock: ARM CNTVCT @ 1000 ticks/us933gitlab # [ 7.860540] redis[796]: Running mode=standalone, port=0.934gitlab # [ 7.861668] redis[796]: Server initialized935gitlab # [ 7.862470] redis[796]: Ready to accept connections unix936gitlab # [ 7.863403] redis[796]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.937gitlab # [ 7.900033] postgresql-pre-start[794]: selecting default "max_connections" ... 100938gitlab # [ 7.955861] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB939gitlab # [ 8.239673] systemd[1]: Started gitlab-workhorse.service.940gitlab # [ 8.454179] gitlab-workhorse[820]: time="2026-10-03T06:59:28Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4941gitlab # [ 8.456481] gitlab-workhorse[820]: time="2026-10-03T06:59:28Z" level=info msg="Using redis/go-redis"942gitlab # [ 8.458191] gitlab-workhorse[820]: time="2026-10-03T06:59:28Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix943gitlab # [ 8.477367] postgresql-pre-start[794]: selecting default time zone ... UTC944gitlab # [ 8.479228] postgresql-pre-start[794]: creating configuration files ... ok945gitlab # [ 8.710046] postgresql-pre-start[794]: running bootstrap script ... ok946gitlab # [ 8.861426] dhcpcd[758]: eth0: soliciting a DHCP lease947gitlab # [ 8.872744] dhcpcd[758]: eth0: offered 10.0.2.15 from 10.0.2.2948gitlab # [ 8.888436] dhcpcd[758]: eth0: probing address 10.0.2.15/24949gitlab # [ 8.942675] dhcpcd[758]: eth0: soliciting an IPv6 router950gitlab # [ 8.944163] dhcpcd[758]: eth0: Router Advertisement from fe80::2951gitlab # [ 8.945556] dhcpcd[758]: eth0: adding address fec0::5054:ff:fe12:3456/64952gitlab # [ 8.947233] dhcpcd[758]: eth0: adding route to fec0::/64953gitlab # [ 8.948560] dhcpcd[758]: eth0: adding default route via fe80::2954gitlab # [ 9.349951] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok955gitlab # [ 9.573172] postgresql-pre-start[794]: syncing data to disk ... ok956gitlab # [ 9.574721] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections957gitlab # [ 9.577022] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.958gitlab # [ 9.580582] postgresql-pre-start[794]: Success. You can now start the database server using:959gitlab # [ 9.582524] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start960gitlab # [ 9.702793] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit961gitlab # [ 9.705888] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432962gitlab # [ 9.707906] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432963gitlab # [ 9.710019] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"964gitlab # [ 9.717727] postgres[863]: [863] LOG: database system was shut down at 2026-10-03 06:59:29 GMT965gitlab # [ 9.729480] postgres[854]: [854] LOG: database system is ready to accept connections966gitlab # [ 9.732655] systemd[1]: Started PostgreSQL Server.967gitlab # [ 9.735727] systemd[1]: Starting PostgreSQL Setup Scripts...968gitlab # [ 10.035026] postgresql-setup-start[874]: CREATE DATABASE969gitlab # [ 10.080109] postgresql-setup-start[879]: CREATE ROLE970gitlab # [ 10.098605] postgresql-setup-start[881]: ALTER DATABASE971gitlab # [ 10.141356] postgresql-setup-start[886]: CREATE ROLE972gitlab # [ 10.144559] systemd[1]: Finished PostgreSQL Setup Scripts.973gitlab # [ 10.146620] systemd[1]: Reached target PostgreSQL.974gitlab # [ 10.150202] systemd[1]: Starting gitlab-postgresql.service...975gitlab # [ 10.343767] gitlab-postgresql-start[894]: CREATE DATABASE976gitlab # [ 10.393910] gitlab-postgresql-start[903]: CREATE EXTENSION977gitlab # [ 10.476646] gitlab-postgresql-start[909]: CREATE EXTENSION978gitlab # [ 10.480814] systemd[1]: Finished gitlab-postgresql.service.979gitlab # [ 10.556861] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[908]: mkdir -p /var/gitlab/state/home/.ssh: OK980gitlab # [ 10.559483] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[908]: chmod 700 /var/gitlab/state/home/.ssh: OK981gitlab # [ 10.618749] systemd[1]: Finished gitlab-config.service.982gitlab # [ 10.622752] systemd[1]: Started gitaly.service.983gitlab # [ 10.626379] systemd[1]: Starting gitlab-db-config.service...984gitlab # [ 11.232907] gitaly[934]: time="2026-10-03T06:59:31.132Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=934985gitlab # [ 11.239103] gitaly[934]: time="2026-10-03T06:59:31.132Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=934986gitlab # [ 11.241816] gitaly[934]: time="2026-10-03T06:59:31.132Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=934987gitlab # [ 11.245219] gitaly[934]: time="2026-10-03T06:59:31.133Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=934988gitlab # [ 11.248049] gitaly[934]: time="2026-10-03T06:59:31.133Z" level=info msg="Starting Gitaly" pid=934 version=989gitlab # [ 11.452321] gitaly[934]: time="2026-10-03T06:59:31.352Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=218 pid=934990gitlab # [ 11.455616] gitaly[934]: time="2026-10-03T06:59:31.355Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=934991gitlab # [ 11.457728] gitaly[934]: time="2026-10-03T06:59:31.355Z" level=info msg="finished initializing command factory" duration_ms=0 pid=934992gitlab # [ 11.459439] gitaly[934]: time="2026-10-03T06:59:31.355Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-934/git-exec-3059951759.d/git pid=934993gitlab # [ 11.467498] gitaly[934]: time="2026-10-03T06:59:31.367Z" level=info msg="finished detecting git version" duration_ms=11 pid=934994gitlab # [ 11.470242] gitaly[934]: time="2026-10-03T06:59:31.367Z" level=info msg="using Git version" pid=934 version=2.53.g995gitlab # [ 11.472437] gitaly[934]: time="2026-10-03T06:59:31.367Z" level=info msg="clearing disk cache object folder" pid=934 storage=default996gitlab # [ 11.475179] gitaly[934]: time="2026-10-03T06:59:31.368Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3032619556 pid=934 storage=default997gitlab # [ 11.478217] gitaly[934]: time="2026-10-03T06:59:31.368Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3032619556 pid=934 storage=default998gitlab # [ 11.481984] gitaly[934]: time="2026-10-03T06:59:31.368Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=934999gitlab # [ 11.485271] gitaly[934]: time="2026-10-03T06:59:31.368Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=9341000gitlab # [ 11.488362] gitaly[934]: time="2026-10-03T06:59:31.368Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3032619556 clear_duration_ms=0 pid=934 storage=default1001gitlab # [ 11.491668] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=9341002gitlab # [ 11.494442] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=934 schema=unix1003gitlab # [ 11.496872] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=info msg="counting repositories" pid=9341004gitlab # [ 11.498482] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=info msg="starting to count repositories" pid=934 storage_path=/var/gitlab/state/repositories1005gitlab # [ 11.500848] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=934 storage_path=/var/gitlab/state/repositories1006gitlab # [ 11.503449] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9341007gitlab # [ 11.505477] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=info msg="listening at address" address=/run/gitaly/gitaly-934/sock.d/intern pid=934 schema=unix1008gitlab # [ 11.508034] gitaly[934]: time="2026-10-03T06:59:31.384Z" level=info msg="finished tempdir cleaner walk" pid=934 storage=default time_ms=01009gitlab # [ 11.510085] gitaly[934]: time="2026-10-03T06:59:31.386Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9341010gitlab # [ 11.512204] gitaly[934]: time="2026-10-03T06:59:31.386Z" level=info msg="Started Gitaly" duration_ms=253 pid=9341011gitlab # [ 11.513995] gitaly[934]: time="2026-10-03T06:59:31.386Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9341012gitlab # [ 11.516126] gitaly[934]: time="2026-10-03T06:59:31.387Z" level=info msg="maintenance: daily scheduled" pid=934 scheduled="2026-10-03 12:00:00 +0000 UTC"1013gitlab # [ 11.821423] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[937]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1014gitlab # [ 11.826270] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[937]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1015gitlab # [ 11.834066] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[937]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1016gitlab # [ 13.256727] dhcpcd[758]: eth0: leased 10.0.2.15 for 86400 seconds1017gitlab # [ 13.264146] dhcpcd[758]: eth0: adding route to 10.0.2.0/241018gitlab # [ 13.265567] dhcpcd[758]: eth0: adding default route via 10.0.2.21019gitlab # [ 13.380511] systemd[1]: Started DHCP Client.1020gitlab # [ 13.382102] systemd[1]: Reached target Network is Online.1021gitlab # [ 14.913237] gitaly[934]: time="2026-10-03T06:59:34.812Z" level=info msg="License database preloaded" duration_ms=3679 pid=9341022gitlab # [ 40.441966] systemd[1]: Starting Logrotate Service...1023gitlab # [ 40.536827] systemd[1]: logrotate.service: Deactivated successfully.1024gitlab # [ 40.538497] systemd[1]: Finished Logrotate Service.1025gitlab # [ 59.970465] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[937]: W, [2026-10-03T07:00:19.870046 #937] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1026gitlab # [ 63.088191] postgres[1477]: [1477] ERROR: relation "feature_gates" does not exist at character 5231027gitlab # [ 63.088463] postgres[1477]: [1477] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1028gitlab # [ 63.088539] postgres[1477]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1029gitlab # [ 63.088609] postgres[1477]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1030gitlab # [ 63.088679] postgres[1477]: attidentity AS identity,1031gitlab # [ 63.088746] postgres[1477]: attgenerated as attgenerated1032gitlab # [ 63.088814] postgres[1477]: FROM pg_attribute a1033gitlab # [ 63.088884] postgres[1477]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1034gitlab # [ 63.088960] postgres[1477]: LEFT JOIN pg_type t ON a.atttypid = t.oid1035gitlab # [ 63.089030] postgres[1477]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1036gitlab # [ 63.089091] postgres[1477]: WHERE a.attrelid = '"feature_gates"'::regclass1037gitlab # [ 63.089148] postgres[1477]: AND a.attnum > 0 AND NOT a.attisdropped1038gitlab # [ 63.089228] postgres[1477]: ORDER BY a.attnum1039gitlab # [ 63.089299] postgres[1477]:1040gitlab # [ 68.366910] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1536]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1041gitlab # [ 68.370509] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1536]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1042gitlab # [ 68.539426] gitlab-workhorse[820]: time="2026-10-03T07:00:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1043gitlab # [ 78.549650] gitlab-workhorse[820]: time="2026-10-03T07:00:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1044gitlab # [ 88.550709] gitlab-workhorse[820]: time="2026-10-03T07:00:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1045gitlab # [ 89.937792] postgres[1631]: [1631] ERROR: relation "table_without_models" does not exist1046gitlab # [ 89.938033] postgres[1631]: [1631] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1047gitlab # [ 98.557459] gitlab-workhorse[820]: time="2026-10-03T07:00:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1048gitlab # [ 107.611554] systemd[1]: Finished gitlab-db-config.service.1049gitlab # [ 107.615544] systemd[1]: Started gitlab-sidekiq.service.1050gitlab # [ 107.620585] systemd[1]: Starting gitlab.service...1051gitlab # [ 108.165660] bundle[1879]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1052gitlab # [ 108.169015] bundle[1879]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1053gitlab # [ 108.172139] bundle[1879]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1054gitlab # [ 108.565498] gitlab-workhorse[820]: time="2026-10-03T07:01:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1055gitlab # [ 114.131005] bundle[1879]: {"timestamp":"2026-10-03T07:01:13.989Z","pid":1879,"message":"* Enabling systemd notification integration"}1056gitlab # [ 114.131410] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"Puma starting in cluster mode..."}1057gitlab # [ 114.131492] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1058gitlab # [ 114.131564] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1059gitlab # [ 114.131643] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Min threads: 0"}1060gitlab # [ 114.131713] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Max threads: 4"}1061gitlab # [ 114.131782] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Environment: production"}1062gitlab # [ 114.131851] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Master PID: 1879"}1063gitlab # [ 114.131938] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Workers: 2"}1064gitlab # [ 114.132007] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.030Z","pid":1879,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1065gitlab # [ 114.132143] bundle[1879]: {"timestamp":"2026-10-03T07:01:14.031Z","pid":1879,"message":"* Preloading application"}1066gitlab # [ 114.701319] sidekiq-cluster[1878]: {"severity":"INFO","time":"2026-10-03T07:01:14.545Z","message":"Starting cluster with 1 processes"}1067gitlab # [ 118.576114] gitlab-workhorse[820]: time="2026-10-03T07:01:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1068gitlab # [ 128.577838] gitlab-workhorse[820]: time="2026-10-03T07:01:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1069gitlab # [ 138.578895] gitlab-workhorse[820]: time="2026-10-03T07:01:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1070gitlab # [ 148.588467] gitlab-workhorse[820]: time="2026-10-03T07:01:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1071gitlab # [ 158.589482] gitlab-workhorse[820]: time="2026-10-03T07:01:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1072gitlab # [ 161.709750] bundle[1879]: W, [2026-10-03T07:02:01.609333 #1879] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1073gitlab # [ 168.164104] sidekiq-cluster[1946]: 2026-10-03T07:02:08.063Z pid=1946 tid=wmq INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1946, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433143808}1074gitlab # [ 168.500499] sidekiq-cluster[1946]: W, [2026-10-03T07:02:08.400773 #1946] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1075gitlab # [ 168.562204] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:08.462Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1076gitlab # [ 168.591365] gitlab-workhorse[820]: time="2026-10-03T07:02:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1077gitlab # [ 172.607962] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.507Z","message":"GitLab reliable fetch activated!"}1078gitlab # [ 172.610804] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.507Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1079gitlab # [ 172.642006] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.542Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1080gitlab # [ 172.645777] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.544Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1081gitlab # [ 172.649622] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.547Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1082gitlab # [ 172.653326] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.550Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1083gitlab # [ 172.657307] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.553Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1084gitlab # [ 172.660932] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.555Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1085gitlab # [ 172.664554] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.557Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1086gitlab # [ 172.667815] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.558Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1087gitlab # [ 172.671309] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.560Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1088gitlab # [ 172.674904] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.562Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1089gitlab # [ 172.678815] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.564Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1090gitlab # [ 172.682570] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.567Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1091gitlab # [ 172.685996] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.569Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1092gitlab # [ 172.689536] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.571Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1093gitlab # [ 172.692813] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.575Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1094gitlab # [ 172.695937] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.578Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1095gitlab # [ 172.699096] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.582Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1096gitlab # [ 172.702052] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.586Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1097gitlab # [ 172.705136] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.590Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1098gitlab # [ 172.707920] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.593Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1099gitlab # [ 172.711133] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.597Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1100gitlab # [ 172.714365] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.599Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1101gitlab # [ 172.717024] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.602Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1102gitlab # [ 172.719612] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.605Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1103gitlab # [ 172.722423] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.608Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1104gitlab # [ 172.725004] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.611Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1105gitlab # [ 172.727369] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.613Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1106gitlab # [ 172.729905] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.616Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1107gitlab # [ 172.732335] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.619Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1108gitlab # [ 172.734732] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.622Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1109gitlab # [ 172.737035] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.625Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1110gitlab # [ 172.739363] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.628Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1111gitlab # [ 172.742473] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.631Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1112gitlab # [ 172.744793] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.634Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1113gitlab # [ 172.747209] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.636Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1114gitlab # [ 172.749803] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.639Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1115gitlab # [ 172.752361] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.641Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1116gitlab # [ 172.754737] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.645Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1117gitlab # [ 172.757218] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.648Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1118gitlab # [ 172.759556] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.650Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1119gitlab # [ 172.762331] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.652Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1120gitlab # [ 172.764879] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.654Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1121gitlab # [ 172.767386] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.656Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1122gitlab # [ 172.769915] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.658Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1123gitlab # [ 172.772514] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.661Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1124gitlab # [ 172.775123] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.663Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1125gitlab # [ 172.777753] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.665Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1126gitlab # [ 172.780312] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.667Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1127gitlab # [ 172.782726] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.669Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1128gitlab # [ 172.785152] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.671Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1129gitlab # [ 172.787380] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.673Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1130gitlab # [ 172.789754] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.676Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1131gitlab # [ 172.791972] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.689Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1132gitlab # [ 172.794618] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.692Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1133gitlab # [ 172.796955] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.694Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1134gitlab # [ 172.799181] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.697Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1135gitlab # [ 172.801518] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.700Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1136gitlab # [ 172.803855] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.703Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1137gitlab # [ 172.806287] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.705Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1138gitlab # [ 172.808642] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.707Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1139gitlab # [ 172.810675] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.709Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1140gitlab # [ 172.812780] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.711Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1141gitlab # [ 172.814921] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.714Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1142gitlab # [ 172.817203] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.717Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1143gitlab # [ 172.819929] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.720Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1144gitlab # [ 172.822697] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.723Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1145gitlab # [ 172.824957] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.725Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1146gitlab # [ 172.827177] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.727Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1147gitlab # [ 172.829650] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.728Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1148gitlab # [ 172.831997] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.730Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1149gitlab # [ 172.834341] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.732Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1150gitlab # [ 172.836673] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.734Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1151gitlab # [ 172.838942] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.736Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1152gitlab # [ 172.841445] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.738Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1153gitlab # [ 172.843816] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.741Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1154gitlab # [ 172.846381] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.742Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1155gitlab # [ 172.848748] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.745Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1156gitlab # [ 172.851259] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.746Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1157gitlab # [ 172.853625] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.749Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1158gitlab # [ 172.855806] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.751Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1159gitlab # [ 172.858378] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.753Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1160gitlab # [ 172.860907] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.755Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1161gitlab # [ 172.863278] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.758Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1162gitlab # [ 172.865663] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.760Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1163gitlab # [ 172.868072] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.762Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1164gitlab # [ 172.870317] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.764Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1165gitlab # [ 172.872751] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.767Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1166gitlab # [ 172.874994] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.769Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1167gitlab # [ 172.877417] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.771Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1168gitlab # [ 172.879741] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.773Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1169gitlab # [ 172.882280] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.775Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1170gitlab # [ 172.884664] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.777Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1171gitlab # [ 172.887019] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.779Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1172gitlab # [ 172.889679] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.781Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1173gitlab # [ 172.891762] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.784Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1174gitlab # [ 172.894387] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.787Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1175gitlab # [ 172.896817] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.789Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1176gitlab # [ 172.899114] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.791Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1177gitlab # [ 172.905560] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.805Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1178gitlab # [ 172.907861] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.808Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1179gitlab # [ 172.910301] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.810Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1180gitlab # [ 172.913797] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.814Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1181gitlab # [ 172.916856] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.817Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1182gitlab # [ 172.919891] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.820Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1183gitlab # [ 172.922928] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.823Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1184gitlab # [ 172.925417] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.825Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1185gitlab # [ 172.927575] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.827Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1186gitlab # [ 172.930189] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.830Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1187gitlab # [ 172.932583] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:12.832Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1188gitlab # [ 178.601439] gitlab-workhorse[820]: time="2026-10-03T07:02:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1189gitlab # [ 188.612781] gitlab-workhorse[820]: time="2026-10-03T07:02:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1190gitlab # [ 198.617708] gitlab-workhorse[820]: time="2026-10-03T07:02:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1191gitlab # [ 208.625649] gitlab-workhorse[820]: time="2026-10-03T07:02:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1192gitlab # [ 210.803641] postgres[2791]: [2791] ERROR: relation "table_without_models" does not exist1193gitlab # [ 210.803893] postgres[2791]: [2791] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1194gitlab # [ 212.793331] postgres[2812]: [2812] ERROR: relation "table_without_models" does not exist1195gitlab # [ 212.793533] postgres[2812]: [2812] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1196gitlab # [ 214.191418] sidekiq-cluster[1946]: InitializerConnections Query: SET client_min_messages TO 'warning'1197gitlab # [ 214.191614] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1198gitlab # [ 214.191658] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1199gitlab # [ 214.191715] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1200gitlab # [ 214.191761] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1201gitlab # [ 214.191802] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1202gitlab # [ 214.191842] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1203gitlab # [ 214.191907] sidekiq-cluster[1946]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1204gitlab # [ 214.193004] sidekiq-cluster[1946]: InitializerConnections Query: SET standard_conforming_strings = on1205gitlab # [ 214.210824] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1206gitlab # [ 214.213437] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1207gitlab # [ 214.215478] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1208gitlab # [ 214.217872] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1209gitlab # [ 214.219849] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1210gitlab # [ 214.221980] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1211gitlab # [ 214.224673] sidekiq-cluster[1946]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1212gitlab # [ 214.228194] sidekiq-cluster[1946]: InitializerConnections Query: SET intervalstyle = iso_86011213gitlab # [ 214.229981] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1214gitlab # [ 214.232313] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1215gitlab # [ 214.234264] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1216gitlab # [ 214.236467] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1217gitlab # [ 214.238251] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1218gitlab # [ 214.240099] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1219gitlab # [ 214.241717] sidekiq-cluster[1946]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1220gitlab # [ 214.265465] sidekiq-cluster[1946]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname1221gitlab # [ 214.268515] sidekiq-cluster[1946]: FROM pg_type as t1222gitlab # [ 214.269615] sidekiq-cluster[1946]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz')1223gitlab # [ 214.272478] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1224gitlab # [ 214.274728] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1225gitlab # [ 214.276755] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1226gitlab # [ 214.278838] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1227gitlab # [ 214.280801] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1228gitlab # [ 214.282690] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1229gitlab # [ 214.285160] sidekiq-cluster[1946]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1230gitlab # [ 214.287687] sidekiq-cluster[1946]: InitializerConnections Query: SET SESSION timezone TO 'UTC'1231gitlab # [ 214.289607] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1232gitlab # [ 214.291855] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1233gitlab # [ 214.293930] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1234gitlab # [ 214.296060] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1235gitlab # [ 214.297964] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1236gitlab # [ 214.299884] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1237gitlab # [ 214.302445] sidekiq-cluster[1946]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1238gitlab # [ 214.305061] sidekiq-cluster[1946]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'sidekiq_throttling_middleware'1239gitlab # [ 214.309781] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value'1240gitlab # [ 214.311842] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature'1241gitlab # [ 214.313688] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value'1242gitlab # [ 214.315531] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?'1243gitlab # [ 214.317237] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?'1244gitlab # [ 214.318853] sidekiq-cluster[1946]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start'1245gitlab # [ 214.320931] sidekiq-cluster[1946]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers1246gitlab # [ 215.870877] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:55.770Z","message":"Booted Rails 7.2.3.1 application in production environment"}1247gitlab # [ 215.874836] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:55.775Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1248gitlab # [ 215.878293] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:55.775Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1249gitlab # [ 215.881426] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:55.775Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1250gitlab # [ 215.884727] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:55.777Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1251gitlab # [ 218.634713] gitlab-workhorse[820]: time="2026-10-03T07:02:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1252gitlab # [ 218.715899] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:58.614Z","message":"Cleaning working queues"}1253gitlab # [ 218.718978] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:58.618Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a9c84b684c7839d43e0ec959","created_at":"2026-10-03T07:01:06.951Z","trace_propagation_headers":{"sentry-trace":"76a38771e14b451b89f5b2d2fd264c0a-87087d6d715c44c0","baggage":"sentry-trace_id=76a38771e14b451b89f5b2d2fd264c0a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b3e005598747841a9bcb8fbc9fb67438","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-03T07:01:06.962Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a9c84b684c7839d43e0ec959: start","job_status":"start","queue_duration_s":111.655868,"scheduling_latency_s":111.656077}1254gitlab # [ 218.989103] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:02:58.888Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a9c84b684c7839d43e0ec959","created_at":"2026-10-03T07:01:06.951Z","trace_propagation_headers":{"sentry-trace":"76a38771e14b451b89f5b2d2fd264c0a-87087d6d715c44c0","baggage":"sentry-trace_id=76a38771e14b451b89f5b2d2fd264c0a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b3e005598747841a9bcb8fbc9fb67438","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-03T07:01:06.962Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a9c84b684c7839d43e0ec959: done: 0.270871 sec","job_status":"done","queue_duration_s":111.655868,"scheduling_latency_s":111.656077,"redis_calls":29,"redis_duration_s":0.007374,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.004266,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001936,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12941434,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.270871,"completed_at":"2026-10-03T07:02:58.888Z","load_balancing_strategy":"primary","db_duration_s":0.067848,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1255gitlab # [ 220.386612] bundle[1879]: {"timestamp":"2026-10-03T07:03:00.286Z","pid":1879,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1256gitlab # [ 220.386906] bundle[1879]: {"timestamp":"2026-10-03T07:03:00.286Z","pid":1879,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1257gitlab # [ 220.387013] bundle[1879]: {"timestamp":"2026-10-03T07:03:00.286Z","pid":1879,"message":"! #\u003cThread:0x0000fae8ffea0540 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1258gitlab # [ 220.387080] bundle[1879]: {"timestamp":"2026-10-03T07:03:00.286Z","pid":1879,"message":"! #\u003cThread:0x0000fae9000a2230 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1259gitlab # [ 220.387144] bundle[1879]: {"timestamp":"2026-10-03T07:03:00.286Z","pid":1879,"message":"Use Ctrl-C to stop"}1260gitlab # [ 220.387204] bundle[1879]: {"timestamp":"2026-10-03T07:03:00.286Z","pid":1879,"message":"Sending status to systemd every 1.0 sec"}1261gitlab # [ 224.001418] bundle[1879]: {"timestamp":"2026-10-03T07:03:03.900Z","pid":1879,"message":"- Worker 0 (PID: 2921) booted in 3.57s, phase: 0"}1262gitlab # [ 224.001708] bundle[1879]: {"timestamp":"2026-10-03T07:03:03.900Z","pid":1879,"message":"- Worker 1 (PID: 2925) booted in 3.54s, phase: 0"}1263gitlab # [ 224.007899] systemd[1]: Started gitlab.service.1264gitlab # [ 224.008234] systemd[1]: Reached target Common target for all GitLab services..1265gitlab # [ 224.011391] systemd[1]: Starting setup-gitlab.service...1266gitlab # [ 224.047102] setup-gitlab-start[2976]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1267gitlab # [ 224.883593] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:03:04 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 8101268gitlab # [ 224.887090] setup-gitlab-start[2976]: + TOKEN=nixbot-test-token-of-20-chars1269gitlab # [ 224.889082] setup-gitlab-start[2976]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1270gitlab # [ 224.902277] runuser[2990]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1271gitlab # [ 225.482758] setup-gitlab-start[2991]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1272gitlab # [ 225.486449] setup-gitlab-start[2991]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1273gitlab # [ 225.489872] setup-gitlab-start[2991]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1274gitlab # [ 226.381890] setup-gitlab-start[2991]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1275gitlab # [ 226.382271] setup-gitlab-start[2991]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1276gitlab # [ 226.388938] setup-gitlab-start[2991]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1277gitlab # [ 232.370783] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:12.270Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e7908988bd83e6dbb4ba4b7c","created_at":"2026-10-03T07:03:12.185Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"894bd0c182eccd34ab104e8daa0f0107","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.265Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e7908988bd83e6dbb4ba4b7c: start","job_status":"start","queue_duration_s":0.004734,"scheduling_latency_s":0.004773}1278gitlab # [ 232.547911] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:12.447Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"e7908988bd83e6dbb4ba4b7c","created_at":"2026-10-03T07:03:12.185Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"894bd0c182eccd34ab104e8daa0f0107","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.265Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-e7908988bd83e6dbb4ba4b7c: done: 0.177707 sec","job_status":"done","queue_duration_s":0.004734,"scheduling_latency_s":0.004773,"redis_calls":16,"redis_duration_s":0.018604000000000002,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.008495,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007898,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002211,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00107178,"duration_s":0.177707,"completed_at":"2026-10-03T07:03:12.447Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.048284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1279gitlab # [ 232.616483] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:12.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"21cd7e87e0accf99cdd34830","created_at":"2026-10-03T07:03:12.277Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02f21495fe4ec24e6264a1c9f1ec695d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.287Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-21cd7e87e0accf99cdd34830: start","job_status":"start","queue_duration_s":0.165277,"scheduling_latency_s":0.165312}1280gitlab # [ 233.238670] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.138Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"21cd7e87e0accf99cdd34830","created_at":"2026-10-03T07:03:12.277Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02f21495fe4ec24e6264a1c9f1ec695d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.287Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-21cd7e87e0accf99cdd34830: done: 0.685969 sec","job_status":"done","queue_duration_s":0.165277,"scheduling_latency_s":0.165312,"redis_calls":565,"redis_duration_s":0.299258,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009605,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.2873,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.373283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030424,"duration_s":0.685969,"completed_at":"2026-10-03T07:03:13.138Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1281gitlab # [ 233.307004] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.140Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0ec3c4695ecc09414f79ab9b","created_at":"2026-10-03T07:03:12.293Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08ac75dce301b006087279c5e279fbfd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.309Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0ec3c4695ecc09414f79ab9b: start","job_status":"start","queue_duration_s":0.830795,"scheduling_latency_s":0.830819}1282gitlab # [ 233.327823] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"0ec3c4695ecc09414f79ab9b","created_at":"2026-10-03T07:03:12.293Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"08ac75dce301b006087279c5e279fbfd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.309Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-0ec3c4695ecc09414f79ab9b: done: 0.046173 sec","job_status":"done","queue_duration_s":0.830795,"scheduling_latency_s":0.830819,"redis_calls":10,"redis_duration_s":0.003947,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044798,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.046173,"completed_at":"2026-10-03T07:03:13.186Z","load_balancing_strategy":"primary","db_duration_s":0.017463,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1283gitlab # [ 233.378931] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5bf0be599907470d5e6cc845","created_at":"2026-10-03T07:03:12.322Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07d17e3a4d8589deb9f7a78ad5428597","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.330Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5bf0be599907470d5e6cc845: start","job_status":"start","queue_duration_s":0.857576,"scheduling_latency_s":0.857599}1284gitlab # [ 233.394317] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.209Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1285gitlab # [ 233.398306] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.218Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"5bf0be599907470d5e6cc845","created_at":"2026-10-03T07:03:12.322Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"07d17e3a4d8589deb9f7a78ad5428597","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.330Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-5bf0be599907470d5e6cc845: done: 0.030279 sec","job_status":"done","queue_duration_s":0.857576,"scheduling_latency_s":0.857599,"redis_calls":10,"redis_duration_s":0.001972,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000353,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032422,"duration_s":0.030279,"completed_at":"2026-10-03T07:03:13.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 233.443492] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.219Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b6a7948bbcae264364459487","created_at":"2026-10-03T07:03:12.346Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d11795b6c9eedab1f4a4ab350d53bf7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-03T07:03:12.361Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b6a7948bbcae264364459487: start","job_status":"start","queue_duration_s":0.857436,"scheduling_latency_s":0.857457}1287gitlab # [ 233.457986] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.232Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1288gitlab # [ 233.461869] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.241Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b6a7948bbcae264364459487","created_at":"2026-10-03T07:03:12.346Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d11795b6c9eedab1f4a4ab350d53bf7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-03T07:03:12.361Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b6a7948bbcae264364459487: done: 0.021715 sec","job_status":"done","queue_duration_s":0.857436,"scheduling_latency_s":0.857457,"redis_calls":10,"redis_duration_s":0.001483,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000317,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002486,"duration_s":0.021715,"completed_at":"2026-10-03T07:03:13.241Z","load_balancing_strategy":"primary","db_duration_s":0.000808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1289gitlab # [ 233.504602] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.242Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5232443ed567d5ec1319a01c","created_at":"2026-10-03T07:03:12.369Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9aaa9e61e7ff21fe6f6d1d1882a220da","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.378Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5232443ed567d5ec1319a01c: start","job_status":"start","queue_duration_s":0.863688,"scheduling_latency_s":0.863709}1290gitlab # [ 233.519491] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.267Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5232443ed567d5ec1319a01c","created_at":"2026-10-03T07:03:12.369Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9aaa9e61e7ff21fe6f6d1d1882a220da","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.378Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5232443ed567d5ec1319a01c: done: 0.024984 sec","job_status":"done","queue_duration_s":0.863688,"scheduling_latency_s":0.863709,"redis_calls":12,"redis_duration_s":0.001959,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000337,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000785,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053226,"exclusive_lock_hold_duration_s":0.0037636,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.024984,"completed_at":"2026-10-03T07:03:13.267Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1291gitlab # [ 233.564648] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fc3feba168a8d43cbd7951a8","created_at":"2026-10-03T07:03:12.395Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a09220541d3f13ce0782c121b1f47c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.408Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fc3feba168a8d43cbd7951a8: start","job_status":"start","queue_duration_s":0.860129,"scheduling_latency_s":0.860149}1292gitlab # [ 233.579548] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.281Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1293gitlab # [ 233.583322] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fc3feba168a8d43cbd7951a8","created_at":"2026-10-03T07:03:12.395Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a09220541d3f13ce0782c121b1f47c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.408Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fc3feba168a8d43cbd7951a8: done: 0.020562 sec","job_status":"done","queue_duration_s":0.860129,"scheduling_latency_s":0.860149,"redis_calls":10,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028434,"duration_s":0.020562,"completed_at":"2026-10-03T07:03:13.289Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000501,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 233.628507] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"03c766ca596f57684350f1e9","created_at":"2026-10-03T07:03:12.444Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4da5730551c47920a514774adb83b7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-03T07:03:12.455Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-03c766ca596f57684350f1e9: start","job_status":"start","queue_duration_s":0.835662,"scheduling_latency_s":0.835683}1295gitlab # [ 233.642091] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"03c766ca596f57684350f1e9","created_at":"2026-10-03T07:03:12.444Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4da5730551c47920a514774adb83b7e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-03T07:03:12.455Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-03c766ca596f57684350f1e9: done: 0.034779 sec","job_status":"done","queue_duration_s":0.835662,"scheduling_latency_s":0.835683,"redis_calls":10,"redis_duration_s":0.001532,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000313,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002601,"duration_s":0.034779,"completed_at":"2026-10-03T07:03:13.325Z","load_balancing_strategy":"primary","db_duration_s":0.004793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 233.684530] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.327Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7369a4f5c6439f4cf5b88576","created_at":"2026-10-03T07:03:12.468Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dab9ee53e7b92f5995b4e93a60fc7450","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.475Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7369a4f5c6439f4cf5b88576: start","job_status":"start","queue_duration_s":0.851783,"scheduling_latency_s":0.851806}1297gitlab # [ 233.699569] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.351Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"7369a4f5c6439f4cf5b88576","created_at":"2026-10-03T07:03:12.468Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dab9ee53e7b92f5995b4e93a60fc7450","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.475Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-7369a4f5c6439f4cf5b88576: done: 0.024025 sec","job_status":"done","queue_duration_s":0.851783,"scheduling_latency_s":0.851806,"redis_calls":12,"redis_duration_s":0.001807,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000656,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000323,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025172,"duration_s":0.024025,"completed_at":"2026-10-03T07:03:13.351Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001245,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 233.743557] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.352Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e3eb04c937013aa11520d5ad","created_at":"2026-10-03T07:03:12.480Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"951d652f4ebad1848512641d047628f2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.487Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e3eb04c937013aa11520d5ad: start","job_status":"start","queue_duration_s":0.864882,"scheduling_latency_s":0.864903}1299gitlab # [ 233.758129] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e3eb04c937013aa11520d5ad","created_at":"2026-10-03T07:03:12.480Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"951d652f4ebad1848512641d047628f2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.487Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e3eb04c937013aa11520d5ad: done: 0.026732 sec","job_status":"done","queue_duration_s":0.864882,"scheduling_latency_s":0.864903,"redis_calls":10,"redis_duration_s":0.001535,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000326,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000261,"duration_s":0.026732,"completed_at":"2026-10-03T07:03:13.379Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003132,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 233.801567] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b5d32cf8da458f5c764e04f6","created_at":"2026-10-03T07:03:12.494Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b81bad2fe121f6147133c98e55845a8b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-03T07:03:12.500Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b5d32cf8da458f5c764e04f6: start","job_status":"start","queue_duration_s":0.87977,"scheduling_latency_s":0.87979}1301gitlab # [ 233.816323] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.392Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1302gitlab # [ 233.819662] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b5d32cf8da458f5c764e04f6","created_at":"2026-10-03T07:03:12.494Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b81bad2fe121f6147133c98e55845a8b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-03T07:03:12.500Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b5d32cf8da458f5c764e04f6: done: 0.020237 sec","job_status":"done","queue_duration_s":0.87977,"scheduling_latency_s":0.87979,"redis_calls":10,"redis_duration_s":0.001491,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027072,"duration_s":0.020237,"completed_at":"2026-10-03T07:03:13.400Z","load_balancing_strategy":"primary","db_duration_s":0.000509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1303gitlab # [ 233.862609] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"95c95d69b4dac87deb1b5596","created_at":"2026-10-03T07:03:12.505Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b5dea71501d9cc131099b44def1ff87","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.510Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-95c95d69b4dac87deb1b5596: start","job_status":"start","queue_duration_s":0.891426,"scheduling_latency_s":0.891447}1304gitlab # [ 233.877833] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.474Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"95c95d69b4dac87deb1b5596","created_at":"2026-10-03T07:03:12.505Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b5dea71501d9cc131099b44def1ff87","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.510Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-95c95d69b4dac87deb1b5596: done: 0.072599 sec","job_status":"done","queue_duration_s":0.891426,"scheduling_latency_s":0.891447,"redis_calls":14,"redis_duration_s":0.002486,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000634,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053942,"exclusive_lock_hold_duration_s":0.04839994,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.072599,"completed_at":"2026-10-03T07:03:13.474Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1305gitlab # [ 233.924329] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1bc35a34a4ed17cc0680c604","created_at":"2026-10-03T07:03:12.514Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fe25de854a5e594da2266f6abfd7083","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.524Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1bc35a34a4ed17cc0680c604: start","job_status":"start","queue_duration_s":0.951706,"scheduling_latency_s":0.951746}1306gitlab # [ 233.939368] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.497Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1bc35a34a4ed17cc0680c604","created_at":"2026-10-03T07:03:12.514Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fe25de854a5e594da2266f6abfd7083","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.524Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1bc35a34a4ed17cc0680c604: done: 0.020634 sec","job_status":"done","queue_duration_s":0.951706,"scheduling_latency_s":0.951746,"redis_calls":10,"redis_duration_s":0.001518,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000307,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000879,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025732,"duration_s":0.020634,"completed_at":"2026-10-03T07:03:13.497Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1307gitlab # [ 233.983485] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.498Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"189f72145e2b76faa6cee4cd","created_at":"2026-10-03T07:03:12.532Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97e6522d8f9770a3c6792d5e99ec8953","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.547Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-189f72145e2b76faa6cee4cd: start","job_status":"start","queue_duration_s":0.950603,"scheduling_latency_s":0.950623}1308gitlab # [ 233.998891] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.547Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"189f72145e2b76faa6cee4cd","created_at":"2026-10-03T07:03:12.532Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"97e6522d8f9770a3c6792d5e99ec8953","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.547Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-189f72145e2b76faa6cee4cd: done: 0.048626 sec","job_status":"done","queue_duration_s":0.950603,"scheduling_latency_s":0.950623,"redis_calls":12,"redis_duration_s":0.0019779999999999997,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058316,"exclusive_lock_hold_duration_s":0.01697796,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.048626,"completed_at":"2026-10-03T07:03:13.547Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1309gitlab # [ 234.046494] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.548Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"83d213d1d80abaa4b346901d","created_at":"2026-10-03T07:03:12.559Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0df93913b0e04516294d1313112cabc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.570Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-83d213d1d80abaa4b346901d: start","job_status":"start","queue_duration_s":0.977912,"scheduling_latency_s":0.977934}1310gitlab # [ 234.061171] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.789Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"83d213d1d80abaa4b346901d","created_at":"2026-10-03T07:03:12.559Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0df93913b0e04516294d1313112cabc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.570Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-83d213d1d80abaa4b346901d: done: 0.240622 sec","job_status":"done","queue_duration_s":0.977912,"scheduling_latency_s":0.977934,"redis_calls":12,"redis_duration_s":0.002296,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000315,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.207444,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052488,"exclusive_lock_hold_duration_s":0.21934114,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.240622,"completed_at":"2026-10-03T07:03:13.789Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1311gitlab # [ 234.106865] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.790Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"af139b387cebfd6ce50c0912","created_at":"2026-10-03T07:03:12.606Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53caaf1d755a72367d6aa1800eb0f8e9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.622Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-af139b387cebfd6ce50c0912: start","job_status":"start","queue_duration_s":1.16886,"scheduling_latency_s":1.168881}1312gitlab # [ 234.121912] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.827Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"af139b387cebfd6ce50c0912","created_at":"2026-10-03T07:03:12.606Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"53caaf1d755a72367d6aa1800eb0f8e9","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.622Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-af139b387cebfd6ce50c0912: done: 0.036584 sec","job_status":"done","queue_duration_s":1.16886,"scheduling_latency_s":1.168881,"redis_calls":13,"redis_duration_s":0.00267,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001387,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01027142,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.036584,"completed_at":"2026-10-03T07:03:13.827Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006438,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1313gitlab # [ 234.167872] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.828Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f3d3662d5ce00e6c6fec9ac7","created_at":"2026-10-03T07:03:12.645Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ead51cf45e3b08ab91db0f5fd8318d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.657Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f3d3662d5ce00e6c6fec9ac7: start","job_status":"start","queue_duration_s":1.171466,"scheduling_latency_s":1.171487}1314gitlab # [ 234.182990] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.841Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1315gitlab # [ 234.186603] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.849Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"f3d3662d5ce00e6c6fec9ac7","created_at":"2026-10-03T07:03:12.645Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ead51cf45e3b08ab91db0f5fd8318d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.657Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-f3d3662d5ce00e6c6fec9ac7: done: 0.020544 sec","job_status":"done","queue_duration_s":1.171466,"scheduling_latency_s":1.171487,"redis_calls":10,"redis_duration_s":0.001556,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025324,"duration_s":0.020544,"completed_at":"2026-10-03T07:03:13.849Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 234.231871] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3f34b51c2c0d93018169fd1b","created_at":"2026-10-03T07:03:12.676Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.687Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3f34b51c2c0d93018169fd1b: start","job_status":"start","queue_duration_s":1.163328,"scheduling_latency_s":1.163347}1317gitlab # [ 234.245847] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"3f34b51c2c0d93018169fd1b","created_at":"2026-10-03T07:03:12.676Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.687Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-3f34b51c2c0d93018169fd1b: done: 0.117103 sec","job_status":"done","queue_duration_s":1.163328,"scheduling_latency_s":1.163347,"redis_calls":71,"redis_duration_s":0.012081,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001516,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000847,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009221,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000497,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000256,"duration_s":0.117103,"completed_at":"2026-10-03T07:03:13.967Z","load_balancing_strategy":"primary","db_duration_s":0.001695,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 234.290292] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f2ea3ee538b00e98ff05210f","created_at":"2026-10-03T07:03:12.701Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c14fb8e33c7ee7097069abc1ab2dd22","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.715Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f2ea3ee538b00e98ff05210f: start","job_status":"start","queue_duration_s":1.253939,"scheduling_latency_s":1.253958}1319gitlab # [ 234.304432] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.996Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f2ea3ee538b00e98ff05210f","created_at":"2026-10-03T07:03:12.701Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c14fb8e33c7ee7097069abc1ab2dd22","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.715Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f2ea3ee538b00e98ff05210f: done: 0.02786 sec","job_status":"done","queue_duration_s":1.253939,"scheduling_latency_s":1.253958,"redis_calls":12,"redis_duration_s":0.00179,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000785,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000694,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004959,"exclusive_lock_hold_duration_s":0.00798746,"duration_s":0.02786,"completed_at":"2026-10-03T07:03:13.996Z","load_balancing_strategy":"primary","db_duration_s":0.003624,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 234.347723] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:13.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5ed8dc1cde707ebe693c0cd9","created_at":"2026-10-03T07:03:12.729Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.741Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5ed8dc1cde707ebe693c0cd9: start","job_status":"start","queue_duration_s":1.256303,"scheduling_latency_s":1.256321}1321gitlab # [ 234.361873] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5ed8dc1cde707ebe693c0cd9","created_at":"2026-10-03T07:03:12.729Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.741Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5ed8dc1cde707ebe693c0cd9: done: 0.03857 sec","job_status":"done","queue_duration_s":1.256303,"scheduling_latency_s":1.256321,"redis_calls":19,"redis_duration_s":0.003423,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000886,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.00023,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001747,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025528,"duration_s":0.03857,"completed_at":"2026-10-03T07:03:14.036Z","load_balancing_strategy":"primary","db_duration_s":0.00227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 234.406234] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"ef968bd3256b8d85aafab4b0","created_at":"2026-10-03T07:03:12.752Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb39f6f8ba2926748c47d96bca0ff826","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.766Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-ef968bd3256b8d85aafab4b0: start","job_status":"start","queue_duration_s":1.27184,"scheduling_latency_s":1.271859}1323gitlab # [ 234.420110] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"ef968bd3256b8d85aafab4b0","created_at":"2026-10-03T07:03:12.752Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb39f6f8ba2926748c47d96bca0ff826","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.766Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-ef968bd3256b8d85aafab4b0: done: 0.030685 sec","job_status":"done","queue_duration_s":1.27184,"scheduling_latency_s":1.271859,"redis_calls":12,"redis_duration_s":0.001844,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000715,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050384,"exclusive_lock_hold_duration_s":0.01088046,"duration_s":0.030685,"completed_at":"2026-10-03T07:03:14.068Z","load_balancing_strategy":"primary","db_duration_s":0.003513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 234.463292] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.069Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a96ac87902b93a20e3414ddd","created_at":"2026-10-03T07:03:12.782Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e23d86876b44b87f26b869a3a182d8d0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.795Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a96ac87902b93a20e3414ddd: start","job_status":"start","queue_duration_s":1.274685,"scheduling_latency_s":1.274703}1325gitlab # [ 234.478227] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.081Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1326gitlab # [ 234.481674] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.089Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a96ac87902b93a20e3414ddd","created_at":"2026-10-03T07:03:12.782Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e23d86876b44b87f26b869a3a182d8d0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-03T07:03:12.795Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a96ac87902b93a20e3414ddd: done: 0.019818 sec","job_status":"done","queue_duration_s":1.274685,"scheduling_latency_s":1.274703,"redis_calls":10,"redis_duration_s":0.001536,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000313,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028592,"duration_s":0.019818,"completed_at":"2026-10-03T07:03:14.089Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1327gitlab # [ 234.525483] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f251a11e69597740cef8192","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f251a11e69597740cef8192: start","job_status":"start","queue_duration_s":0.132817,"scheduling_latency_s":0.132837}1328gitlab # [ 234.538448] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f251a11e69597740cef8192","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f251a11e69597740cef8192: done: 0.02336 sec","job_status":"done","queue_duration_s":0.132817,"scheduling_latency_s":0.132837,"redis_calls":16,"redis_duration_s":0.002915,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001915,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027188,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02336,"completed_at":"2026-10-03T07:03:14.114Z","load_balancing_strategy":"primary","db_duration_s":0.000329,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1329gitlab # [ 234.580812] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d2c121a0743e8227f32ade","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d2c121a0743e8227f32ade: start","job_status":"start","queue_duration_s":0.15742,"scheduling_latency_s":0.15744}1330gitlab # [ 234.593702] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d2c121a0743e8227f32ade","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d2c121a0743e8227f32ade: done: 0.016096 sec","job_status":"done","queue_duration_s":0.15742,"scheduling_latency_s":0.15744,"redis_calls":13,"redis_duration_s":0.001657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016096,"completed_at":"2026-10-03T07:03:14.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1331gitlab # [ 234.635027] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e321807e06b7158e432e3c","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e321807e06b7158e432e3c: start","job_status":"start","queue_duration_s":0.174934,"scheduling_latency_s":0.174954}1332gitlab # [ 234.648124] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e321807e06b7158e432e3c","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e321807e06b7158e432e3c: done: 0.016161 sec","job_status":"done","queue_duration_s":0.174934,"scheduling_latency_s":0.174954,"redis_calls":13,"redis_duration_s":0.0019199999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016161,"completed_at":"2026-10-03T07:03:14.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1333gitlab # [ 234.688866] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.150Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fedaa0ee8b278cd9fb680e1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fedaa0ee8b278cd9fb680e1: start","job_status":"start","queue_duration_s":0.192238,"scheduling_latency_s":0.192257}1334gitlab # [ 234.701796] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fedaa0ee8b278cd9fb680e1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fedaa0ee8b278cd9fb680e1: done: 0.015912 sec","job_status":"done","queue_duration_s":0.192238,"scheduling_latency_s":0.192257,"redis_calls":13,"redis_duration_s":0.001733,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015912,"completed_at":"2026-10-03T07:03:14.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1335gitlab # [ 234.742110] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e70d8d02076767c9d4fefc2b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e70d8d02076767c9d4fefc2b: start","job_status":"start","queue_duration_s":0.209359,"scheduling_latency_s":0.209378}1336gitlab # [ 234.755150] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e70d8d02076767c9d4fefc2b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e70d8d02076767c9d4fefc2b: done: 0.015869 sec","job_status":"done","queue_duration_s":0.209359,"scheduling_latency_s":0.209378,"redis_calls":13,"redis_duration_s":0.001709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015869,"completed_at":"2026-10-03T07:03:14.183Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1337gitlab # [ 234.795665] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a072b43e32076a46ef1b5a8b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a072b43e32076a46ef1b5a8b: start","job_status":"start","queue_duration_s":0.226425,"scheduling_latency_s":0.226445}1338gitlab # [ 234.808654] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a072b43e32076a46ef1b5a8b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a072b43e32076a46ef1b5a8b: done: 0.015801 sec","job_status":"done","queue_duration_s":0.226425,"scheduling_latency_s":0.226445,"redis_calls":13,"redis_duration_s":0.00165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015801,"completed_at":"2026-10-03T07:03:14.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1339gitlab # [ 234.850002] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18c1c455876ee69052f6212b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18c1c455876ee69052f6212b: start","job_status":"start","queue_duration_s":0.243643,"scheduling_latency_s":0.243662}1340gitlab # [ 234.862938] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"18c1c455876ee69052f6212b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-18c1c455876ee69052f6212b: done: 0.016076 sec","job_status":"done","queue_duration_s":0.243643,"scheduling_latency_s":0.243662,"redis_calls":13,"redis_duration_s":0.001761,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016076,"completed_at":"2026-10-03T07:03:14.217Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1341gitlab # [ 234.903655] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76719255c85eae48dc02e8e9","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76719255c85eae48dc02e8e9: start","job_status":"start","queue_duration_s":0.261283,"scheduling_latency_s":0.261303}1342gitlab # [ 234.917225] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76719255c85eae48dc02e8e9","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76719255c85eae48dc02e8e9: done: 0.018249 sec","job_status":"done","queue_duration_s":0.261283,"scheduling_latency_s":0.261303,"redis_calls":13,"redis_duration_s":0.002139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018249,"completed_at":"2026-10-03T07:03:14.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1343gitlab # [ 234.959015] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26a72521695c065045bbc2d1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26a72521695c065045bbc2d1: start","job_status":"start","queue_duration_s":0.280957,"scheduling_latency_s":0.280977}1344gitlab # [ 234.972446] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26a72521695c065045bbc2d1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26a72521695c065045bbc2d1: done: 0.015983 sec","job_status":"done","queue_duration_s":0.280957,"scheduling_latency_s":0.280977,"redis_calls":13,"redis_duration_s":0.001803,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000633,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015983,"completed_at":"2026-10-03T07:03:14.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1345gitlab # [ 235.014110] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"375041a96573da9e3b607522","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-375041a96573da9e3b607522: start","job_status":"start","queue_duration_s":0.298316,"scheduling_latency_s":0.298336}1346gitlab # [ 235.027424] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"375041a96573da9e3b607522","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-375041a96573da9e3b607522: done: 0.015589 sec","job_status":"done","queue_duration_s":0.298316,"scheduling_latency_s":0.298336,"redis_calls":13,"redis_duration_s":0.0017399999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015589,"completed_at":"2026-10-03T07:03:14.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1347gitlab # [ 235.070230] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d34393a91e8497a4d759a12b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d34393a91e8497a4d759a12b: start","job_status":"start","queue_duration_s":0.315467,"scheduling_latency_s":0.315486}1348gitlab # [ 235.082951] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d34393a91e8497a4d759a12b","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d34393a91e8497a4d759a12b: done: 0.015461 sec","job_status":"done","queue_duration_s":0.315467,"scheduling_latency_s":0.315486,"redis_calls":13,"redis_duration_s":0.0016519999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015461,"completed_at":"2026-10-03T07:03:14.288Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1349gitlab # [ 235.124138] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27745c539d4e40186ac06d15","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27745c539d4e40186ac06d15: start","job_status":"start","queue_duration_s":0.332125,"scheduling_latency_s":0.332145}1350gitlab # [ 235.137223] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27745c539d4e40186ac06d15","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27745c539d4e40186ac06d15: done: 0.015523 sec","job_status":"done","queue_duration_s":0.332125,"scheduling_latency_s":0.332145,"redis_calls":13,"redis_duration_s":0.001859,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001296,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015523,"completed_at":"2026-10-03T07:03:14.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1351gitlab # [ 235.177980] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16626f6316fc8efc6c9b6c9e","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16626f6316fc8efc6c9b6c9e: start","job_status":"start","queue_duration_s":0.349056,"scheduling_latency_s":0.349075}1352gitlab # [ 235.190883] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.322Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16626f6316fc8efc6c9b6c9e","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16626f6316fc8efc6c9b6c9e: done: 0.015938 sec","job_status":"done","queue_duration_s":0.349056,"scheduling_latency_s":0.349075,"redis_calls":13,"redis_duration_s":0.002023,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001406,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015938,"completed_at":"2026-10-03T07:03:14.322Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1353gitlab # [ 235.231864] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"934e0182a8db7ea2421cc9ea","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-934e0182a8db7ea2421cc9ea: start","job_status":"start","queue_duration_s":0.366174,"scheduling_latency_s":0.366193}1354gitlab # [ 235.245486] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"934e0182a8db7ea2421cc9ea","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-934e0182a8db7ea2421cc9ea: done: 0.01614 sec","job_status":"done","queue_duration_s":0.366174,"scheduling_latency_s":0.366193,"redis_calls":13,"redis_duration_s":0.0019039999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01614,"completed_at":"2026-10-03T07:03:14.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1355gitlab # [ 235.286230] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e7c319b4fe644317e0fae5","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e7c319b4fe644317e0fae5: start","job_status":"start","queue_duration_s":0.383563,"scheduling_latency_s":0.383581}1356gitlab # [ 235.298979] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.357Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e7c319b4fe644317e0fae5","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e7c319b4fe644317e0fae5: done: 0.015968 sec","job_status":"done","queue_duration_s":0.383563,"scheduling_latency_s":0.383581,"redis_calls":13,"redis_duration_s":0.001748,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000632,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015968,"completed_at":"2026-10-03T07:03:14.357Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1357gitlab # [ 235.339641] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a37e1d9f687172e45b491979","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a37e1d9f687172e45b491979: start","job_status":"start","queue_duration_s":0.400982,"scheduling_latency_s":0.401001}1358gitlab # [ 235.352725] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a37e1d9f687172e45b491979","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a37e1d9f687172e45b491979: done: 0.015746 sec","job_status":"done","queue_duration_s":0.400982,"scheduling_latency_s":0.401001,"redis_calls":13,"redis_duration_s":0.001734,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015746,"completed_at":"2026-10-03T07:03:14.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1359gitlab # [ 235.393383] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5fa7f82128f4c987ba57643","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5fa7f82128f4c987ba57643: start","job_status":"start","queue_duration_s":0.418065,"scheduling_latency_s":0.418083}1360gitlab # [ 235.406568] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5fa7f82128f4c987ba57643","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5fa7f82128f4c987ba57643: done: 0.015648 sec","job_status":"done","queue_duration_s":0.418065,"scheduling_latency_s":0.418083,"redis_calls":13,"redis_duration_s":0.001735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015648,"completed_at":"2026-10-03T07:03:14.391Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1361gitlab # [ 235.448045] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4cf407cdce5a7e6f4dfabb","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4cf407cdce5a7e6f4dfabb: start","job_status":"start","queue_duration_s":0.435069,"scheduling_latency_s":0.435088}1362gitlab # [ 235.460676] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4cf407cdce5a7e6f4dfabb","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4cf407cdce5a7e6f4dfabb: done: 0.015864 sec","job_status":"done","queue_duration_s":0.435069,"scheduling_latency_s":0.435088,"redis_calls":13,"redis_duration_s":0.00178,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001127,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015864,"completed_at":"2026-10-03T07:03:14.408Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1363gitlab # [ 235.501040] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb0d145aa78a7dbf3041f26a","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb0d145aa78a7dbf3041f26a: start","job_status":"start","queue_duration_s":0.45233,"scheduling_latency_s":0.45235}1364gitlab # [ 235.513907] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb0d145aa78a7dbf3041f26a","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb0d145aa78a7dbf3041f26a: done: 0.015544 sec","job_status":"done","queue_duration_s":0.45233,"scheduling_latency_s":0.45235,"redis_calls":13,"redis_duration_s":0.0016289999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015544,"completed_at":"2026-10-03T07:03:14.425Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1365gitlab # [ 235.554342] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51cf1f1cdde5c4f02ce68a47","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51cf1f1cdde5c4f02ce68a47: start","job_status":"start","queue_duration_s":0.469181,"scheduling_latency_s":0.4692}1366gitlab # [ 235.566996] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51cf1f1cdde5c4f02ce68a47","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51cf1f1cdde5c4f02ce68a47: done: 0.015859 sec","job_status":"done","queue_duration_s":0.469181,"scheduling_latency_s":0.4692,"redis_calls":13,"redis_duration_s":0.0020109999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001362,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015859,"completed_at":"2026-10-03T07:03:14.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1367gitlab # [ 235.607886] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2de2981848a4aa2fa8cd23f","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2de2981848a4aa2fa8cd23f: start","job_status":"start","queue_duration_s":0.486439,"scheduling_latency_s":0.486459}1368gitlab # [ 235.620690] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2de2981848a4aa2fa8cd23f","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2de2981848a4aa2fa8cd23f: done: 0.015873 sec","job_status":"done","queue_duration_s":0.486439,"scheduling_latency_s":0.486459,"redis_calls":13,"redis_duration_s":0.001732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011298,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015873,"completed_at":"2026-10-03T07:03:14.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1369gitlab # [ 235.662151] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"246308379377948b0d1cb6b1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-246308379377948b0d1cb6b1: start","job_status":"start","queue_duration_s":0.503486,"scheduling_latency_s":0.503505}1370gitlab # [ 235.675136] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"246308379377948b0d1cb6b1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-246308379377948b0d1cb6b1: done: 0.016299 sec","job_status":"done","queue_duration_s":0.503486,"scheduling_latency_s":0.503505,"redis_calls":13,"redis_duration_s":0.00164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016299,"completed_at":"2026-10-03T07:03:14.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1371gitlab # [ 235.715722] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40498eded1a6e08ed51000ce","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40498eded1a6e08ed51000ce: start","job_status":"start","queue_duration_s":0.520926,"scheduling_latency_s":0.520944}1372gitlab # [ 235.728558] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"40498eded1a6e08ed51000ce","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-40498eded1a6e08ed51000ce: done: 0.016402 sec","job_status":"done","queue_duration_s":0.520926,"scheduling_latency_s":0.520944,"redis_calls":13,"redis_duration_s":0.0018110000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016402,"completed_at":"2026-10-03T07:03:14.495Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1373gitlab # [ 235.769139] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adf8a6898e34c01ae16e0f6f","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adf8a6898e34c01ae16e0f6f: start","job_status":"start","queue_duration_s":0.538515,"scheduling_latency_s":0.538568}1374gitlab # [ 235.781937] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adf8a6898e34c01ae16e0f6f","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adf8a6898e34c01ae16e0f6f: done: 0.015963 sec","job_status":"done","queue_duration_s":0.538515,"scheduling_latency_s":0.538568,"redis_calls":13,"redis_duration_s":0.001706,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015963,"completed_at":"2026-10-03T07:03:14.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 235.822338] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9781d06c9366698adcd3491d","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9781d06c9366698adcd3491d: start","job_status":"start","queue_duration_s":0.5557,"scheduling_latency_s":0.555719}1376gitlab # [ 235.835255] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9781d06c9366698adcd3491d","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9781d06c9366698adcd3491d: done: 0.015908 sec","job_status":"done","queue_duration_s":0.5557,"scheduling_latency_s":0.555719,"redis_calls":13,"redis_duration_s":0.0021019999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001269,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015908,"completed_at":"2026-10-03T07:03:14.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 235.876753] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cae9a71444039fee9ced85c5","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cae9a71444039fee9ced85c5: start","job_status":"start","queue_duration_s":0.572964,"scheduling_latency_s":0.572983}1378gitlab # [ 235.889592] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cae9a71444039fee9ced85c5","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cae9a71444039fee9ced85c5: done: 0.130583 sec","job_status":"done","queue_duration_s":0.572964,"scheduling_latency_s":0.572983,"redis_calls":13,"redis_duration_s":0.001718,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124381,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.130583,"completed_at":"2026-10-03T07:03:14.661Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1379gitlab # [ 235.929877] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48afc26ed1f63c271521ac05","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48afc26ed1f63c271521ac05: start","job_status":"start","queue_duration_s":0.704921,"scheduling_latency_s":0.704942}1380gitlab # [ 235.942519] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48afc26ed1f63c271521ac05","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48afc26ed1f63c271521ac05: done: 0.016765 sec","job_status":"done","queue_duration_s":0.704921,"scheduling_latency_s":0.704942,"redis_calls":13,"redis_duration_s":0.001826,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000656,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016765,"completed_at":"2026-10-03T07:03:14.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 235.982938] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9be1213fa689d21adba574a0","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9be1213fa689d21adba574a0: start","job_status":"start","queue_duration_s":0.72289,"scheduling_latency_s":0.722909}1382gitlab # [ 235.996050] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9be1213fa689d21adba574a0","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9be1213fa689d21adba574a0: done: 0.014423 sec","job_status":"done","queue_duration_s":0.72289,"scheduling_latency_s":0.722909,"redis_calls":13,"redis_duration_s":0.001751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014423,"completed_at":"2026-10-03T07:03:14.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 236.036591] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a071115a9be040dc749227ab","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a071115a9be040dc749227ab: start","job_status":"start","queue_duration_s":0.738509,"scheduling_latency_s":0.738528}1384gitlab # [ 236.049546] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a071115a9be040dc749227ab","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a071115a9be040dc749227ab: done: 0.015253 sec","job_status":"done","queue_duration_s":0.738509,"scheduling_latency_s":0.738528,"redis_calls":13,"redis_duration_s":0.001664,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015253,"completed_at":"2026-10-03T07:03:14.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 236.090724] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdfdb2f01e7b9714f934510e","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdfdb2f01e7b9714f934510e: start","job_status":"start","queue_duration_s":0.755643,"scheduling_latency_s":0.755663}1386gitlab # [ 236.103441] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdfdb2f01e7b9714f934510e","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdfdb2f01e7b9714f934510e: done: 0.015397 sec","job_status":"done","queue_duration_s":0.755643,"scheduling_latency_s":0.755663,"redis_calls":13,"redis_duration_s":0.0017369999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015397,"completed_at":"2026-10-03T07:03:14.728Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1387gitlab # [ 236.144528] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d9ee829b50673dbef7b239","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d9ee829b50673dbef7b239: start","job_status":"start","queue_duration_s":0.772207,"scheduling_latency_s":0.772225}1388gitlab # [ 236.157103] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9d9ee829b50673dbef7b239","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9d9ee829b50673dbef7b239: done: 0.015531 sec","job_status":"done","queue_duration_s":0.772207,"scheduling_latency_s":0.772225,"redis_calls":13,"redis_duration_s":0.0019490000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001358,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015531,"completed_at":"2026-10-03T07:03:14.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 236.197583] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46f74b3046cbd98601224f28","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46f74b3046cbd98601224f28: start","job_status":"start","queue_duration_s":0.788878,"scheduling_latency_s":0.788898}1390gitlab # [ 236.210497] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46f74b3046cbd98601224f28","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46f74b3046cbd98601224f28: done: 0.015624 sec","job_status":"done","queue_duration_s":0.788878,"scheduling_latency_s":0.788898,"redis_calls":13,"redis_duration_s":0.001711,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010969,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015624,"completed_at":"2026-10-03T07:03:14.762Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 236.250914] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d589dfbb33dc4f098a805ec1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d589dfbb33dc4f098a805ec1: start","job_status":"start","queue_duration_s":0.805749,"scheduling_latency_s":0.805769}1392gitlab # [ 236.264598] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d589dfbb33dc4f098a805ec1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d589dfbb33dc4f098a805ec1: done: 0.015458 sec","job_status":"done","queue_duration_s":0.805749,"scheduling_latency_s":0.805769,"redis_calls":13,"redis_duration_s":0.001693,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015458,"completed_at":"2026-10-03T07:03:14.779Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 236.305154] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eea6c320a30b61e617136393","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eea6c320a30b61e617136393: start","job_status":"start","queue_duration_s":0.82236,"scheduling_latency_s":0.82238}1394gitlab # [ 236.317741] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eea6c320a30b61e617136393","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eea6c320a30b61e617136393: done: 0.016135 sec","job_status":"done","queue_duration_s":0.82236,"scheduling_latency_s":0.82238,"redis_calls":13,"redis_duration_s":0.0016879999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001086,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016135,"completed_at":"2026-10-03T07:03:14.796Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1395gitlab # [ 236.358147] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"042186a86fee39cbdae88a8d","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-042186a86fee39cbdae88a8d: start","job_status":"start","queue_duration_s":0.839692,"scheduling_latency_s":0.83971}1396gitlab # [ 236.371041] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"042186a86fee39cbdae88a8d","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-042186a86fee39cbdae88a8d: done: 0.015681 sec","job_status":"done","queue_duration_s":0.839692,"scheduling_latency_s":0.83971,"redis_calls":13,"redis_duration_s":0.001716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015681,"completed_at":"2026-10-03T07:03:14.813Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 236.411963] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efddc86b9a99552f6fc147ee","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efddc86b9a99552f6fc147ee: start","job_status":"start","queue_duration_s":0.856545,"scheduling_latency_s":0.856564}1398gitlab # [ 236.425013] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efddc86b9a99552f6fc147ee","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efddc86b9a99552f6fc147ee: done: 0.015421 sec","job_status":"done","queue_duration_s":0.856545,"scheduling_latency_s":0.856564,"redis_calls":13,"redis_duration_s":0.001642,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000609,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001033,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015421,"completed_at":"2026-10-03T07:03:14.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1399gitlab # [ 236.465982] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7d69cafd7d26a8ed7429525","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7d69cafd7d26a8ed7429525: start","job_status":"start","queue_duration_s":0.873186,"scheduling_latency_s":0.873204}1400gitlab # [ 236.478775] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7d69cafd7d26a8ed7429525","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7d69cafd7d26a8ed7429525: done: 0.015838 sec","job_status":"done","queue_duration_s":0.873186,"scheduling_latency_s":0.873204,"redis_calls":13,"redis_duration_s":0.001752,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015838,"completed_at":"2026-10-03T07:03:14.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1401gitlab # [ 236.519347] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08b36e827a0980b33aa19a53","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08b36e827a0980b33aa19a53: start","job_status":"start","queue_duration_s":0.890203,"scheduling_latency_s":0.890244}1402gitlab # [ 236.532321] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08b36e827a0980b33aa19a53","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08b36e827a0980b33aa19a53: done: 0.0153 sec","job_status":"done","queue_duration_s":0.890203,"scheduling_latency_s":0.890244,"redis_calls":13,"redis_duration_s":0.0016450000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0153,"completed_at":"2026-10-03T07:03:14.863Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 236.573152] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8b05de3ff37a54a5d46e3ec","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8b05de3ff37a54a5d46e3ec: start","job_status":"start","queue_duration_s":0.907419,"scheduling_latency_s":0.907438}1404gitlab # [ 236.586017] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8b05de3ff37a54a5d46e3ec","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8b05de3ff37a54a5d46e3ec: done: 0.015478 sec","job_status":"done","queue_duration_s":0.907419,"scheduling_latency_s":0.907438,"redis_calls":13,"redis_duration_s":0.002455,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001345,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010823,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015478,"completed_at":"2026-10-03T07:03:14.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1405gitlab # [ 236.626215] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f37e70a106eb7059a1e37777","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f37e70a106eb7059a1e37777: start","job_status":"start","queue_duration_s":0.924229,"scheduling_latency_s":0.924249}1406gitlab # [ 236.639128] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f37e70a106eb7059a1e37777","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f37e70a106eb7059a1e37777: done: 0.015373 sec","job_status":"done","queue_duration_s":0.924229,"scheduling_latency_s":0.924249,"redis_calls":13,"redis_duration_s":0.0016849999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015373,"completed_at":"2026-10-03T07:03:14.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 236.680613] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbfabe2c4577a3146b7542ad","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbfabe2c4577a3146b7542ad: start","job_status":"start","queue_duration_s":0.940761,"scheduling_latency_s":0.940779}1408gitlab # [ 236.693271] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbfabe2c4577a3146b7542ad","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbfabe2c4577a3146b7542ad: done: 0.01532 sec","job_status":"done","queue_duration_s":0.940761,"scheduling_latency_s":0.940779,"redis_calls":13,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01532,"completed_at":"2026-10-03T07:03:14.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 236.733659] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaa736f852b82e3d1afed2ce","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaa736f852b82e3d1afed2ce: start","job_status":"start","queue_duration_s":0.957428,"scheduling_latency_s":0.957448}1410gitlab # [ 236.746522] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aaa736f852b82e3d1afed2ce","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aaa736f852b82e3d1afed2ce: done: 0.015399 sec","job_status":"done","queue_duration_s":0.957428,"scheduling_latency_s":0.957448,"redis_calls":13,"redis_duration_s":0.001678,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015399,"completed_at":"2026-10-03T07:03:14.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 236.786920] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2752707198914c143f34acda","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2752707198914c143f34acda: start","job_status":"start","queue_duration_s":0.974833,"scheduling_latency_s":0.974852}1412gitlab # [ 236.799904] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2752707198914c143f34acda","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2752707198914c143f34acda: done: 0.016216 sec","job_status":"done","queue_duration_s":0.974833,"scheduling_latency_s":0.974852,"redis_calls":13,"redis_duration_s":0.001714,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016216,"completed_at":"2026-10-03T07:03:14.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 236.840607] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b0d91b5438b6535af6db3a7","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b0d91b5438b6535af6db3a7: start","job_status":"start","queue_duration_s":0.992218,"scheduling_latency_s":0.992236}1414gitlab # [ 236.853474] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b0d91b5438b6535af6db3a7","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b0d91b5438b6535af6db3a7: done: 0.015576 sec","job_status":"done","queue_duration_s":0.992218,"scheduling_latency_s":0.992236,"redis_calls":13,"redis_duration_s":0.0016589999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015576,"completed_at":"2026-10-03T07:03:14.965Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 236.894738] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc069f752b9858a7b1f4a798","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc069f752b9858a7b1f4a798: start","job_status":"start","queue_duration_s":1.008971,"scheduling_latency_s":1.008991}1416gitlab # [ 236.908185] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc069f752b9858a7b1f4a798","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc069f752b9858a7b1f4a798: done: 0.01511 sec","job_status":"done","queue_duration_s":1.008971,"scheduling_latency_s":1.008991,"redis_calls":13,"redis_duration_s":0.0016679999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01511,"completed_at":"2026-10-03T07:03:14.982Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 236.950336] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:14.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0836eb59d5922ff4a854289","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0836eb59d5922ff4a854289: start","job_status":"start","queue_duration_s":1.025242,"scheduling_latency_s":1.025261}1418gitlab # [ 236.963618] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.001Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0836eb59d5922ff4a854289","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0836eb59d5922ff4a854289: done: 0.01839 sec","job_status":"done","queue_duration_s":1.025242,"scheduling_latency_s":1.025261,"redis_calls":13,"redis_duration_s":0.0016099999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01839,"completed_at":"2026-10-03T07:03:15.001Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 237.004751] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6f1994fac8760b9fc89604e","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6f1994fac8760b9fc89604e: start","job_status":"start","queue_duration_s":1.044814,"scheduling_latency_s":1.044832}1420gitlab # [ 237.017539] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6f1994fac8760b9fc89604e","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6f1994fac8760b9fc89604e: done: 0.016524 sec","job_status":"done","queue_duration_s":1.044814,"scheduling_latency_s":1.044832,"redis_calls":13,"redis_duration_s":0.002887,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016524,"completed_at":"2026-10-03T07:03:15.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 237.058849] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a18bc3bf5995568ee4310f1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a18bc3bf5995568ee4310f1: start","job_status":"start","queue_duration_s":1.062612,"scheduling_latency_s":1.062631}1422gitlab # [ 237.071940] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a18bc3bf5995568ee4310f1","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a18bc3bf5995568ee4310f1: done: 0.016183 sec","job_status":"done","queue_duration_s":1.062612,"scheduling_latency_s":1.062631,"redis_calls":13,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016183,"completed_at":"2026-10-03T07:03:15.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1423gitlab # [ 237.113309] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0a48e4cbf6364d2fe2a8577","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0a48e4cbf6364d2fe2a8577: start","job_status":"start","queue_duration_s":1.079942,"scheduling_latency_s":1.07996}1424gitlab # [ 237.126395] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0a48e4cbf6364d2fe2a8577","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0a48e4cbf6364d2fe2a8577: done: 0.015705 sec","job_status":"done","queue_duration_s":1.079942,"scheduling_latency_s":1.07996,"redis_calls":13,"redis_duration_s":0.0016560000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015705,"completed_at":"2026-10-03T07:03:15.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 237.167443] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8b5563941ed1690adc1d197","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8b5563941ed1690adc1d197: start","job_status":"start","queue_duration_s":1.097545,"scheduling_latency_s":1.097564}1426gitlab # [ 237.180544] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8b5563941ed1690adc1d197","created_at":"2026-10-03T07:03:13.871Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8c6e4a7b982840a8","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"70c57fede452e4f5e0c76444ad360d8d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:13.957Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8b5563941ed1690adc1d197: done: 0.015844 sec","job_status":"done","queue_duration_s":1.097545,"scheduling_latency_s":1.097564,"redis_calls":13,"redis_duration_s":0.001703,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000628,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015844,"completed_at":"2026-10-03T07:03:15.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 237.221280] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.072Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"933e87d6852929643e59990c","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-933e87d6852929643e59990c: start","job_status":"start","queue_duration_s":1.043887,"scheduling_latency_s":1.043905}1428gitlab # [ 237.233994] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.091Z","message":"No existing merge request to be cleaned up."}1429gitlab # [ 237.235731] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.105Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"933e87d6852929643e59990c","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-933e87d6852929643e59990c: done: 0.033176 sec","job_status":"done","queue_duration_s":1.043887,"scheduling_latency_s":1.043905,"redis_calls":13,"redis_duration_s":0.0020239999999999998,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000344,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031336,"duration_s":0.033176,"completed_at":"2026-10-03T07:03:15.105Z","load_balancing_strategy":"primary","db_duration_s":0.004911,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 237.277862] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.106Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"092a82d3e9059c45da66be33","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-092a82d3e9059c45da66be33: start","job_status":"start","queue_duration_s":1.078255,"scheduling_latency_s":1.078274}1431gitlab # [ 237.291443] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.118Z","message":"No existing merge request to be cleaned up."}1432gitlab # [ 237.293246] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.126Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"092a82d3e9059c45da66be33","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-092a82d3e9059c45da66be33: done: 0.019728 sec","job_status":"done","queue_duration_s":1.078255,"scheduling_latency_s":1.078274,"redis_calls":10,"redis_duration_s":0.001633,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000717,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019728,"completed_at":"2026-10-03T07:03:15.126Z","load_balancing_strategy":"primary","db_duration_s":0.001825,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 237.333019] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.128Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2587cf5e822209b719b764a8","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2587cf5e822209b719b764a8: start","job_status":"start","queue_duration_s":1.100177,"scheduling_latency_s":1.100197}1434gitlab # [ 237.345931] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.139Z","message":"No existing merge request to be cleaned up."}1435gitlab # [ 237.347694] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.148Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2587cf5e822209b719b764a8","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2587cf5e822209b719b764a8: done: 0.01979 sec","job_status":"done","queue_duration_s":1.100177,"scheduling_latency_s":1.100197,"redis_calls":10,"redis_duration_s":0.001761,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000856,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01979,"completed_at":"2026-10-03T07:03:15.148Z","load_balancing_strategy":"primary","db_duration_s":0.001566,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1436gitlab # [ 237.387938] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.149Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dec318d0083fdff0cf4b5574","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dec318d0083fdff0cf4b5574: start","job_status":"start","queue_duration_s":1.121135,"scheduling_latency_s":1.121155}1437gitlab # [ 237.401124] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.161Z","message":"No existing merge request to be cleaned up."}1438gitlab # [ 237.402829] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:03:15.169Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dec318d0083fdff0cf4b5574","created_at":"2026-10-03T07:03:14.015Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-8bd0dccc963b43ea","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"1a94edb9f84829baa71a18f60f1508c7","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:03:14.028Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dec318d0083fdff0cf4b5574: done: 0.019598 sec","job_status":"done","queue_duration_s":1.121135,"scheduling_latency_s":1.121155,"redis_calls":10,"redis_duration_s":0.001519,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000847,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019598,"completed_at":"2026-10-03T07:03:15.169Z","load_balancing_strategy":"primary","db_duration_s":0.001596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 271.746043] setup-gitlab-start[2991]: W, [2026-10-03T07:03:51.645629 #2991] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1440gitlab # [ 282.234429] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:02.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0f2a52302ce8b6d46efa083f","created_at":"2026-10-03T07:04:02.115Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba186ca06e90dee35018f1e6b7b56508","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.130Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0f2a52302ce8b6d46efa083f: start","job_status":"start","queue_duration_s":0.002465,"scheduling_latency_s":0.002506}1441gitlab # [ 282.302406] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:02.201Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"0f2a52302ce8b6d46efa083f","created_at":"2026-10-03T07:04:02.115Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ba186ca06e90dee35018f1e6b7b56508","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.130Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-0f2a52302ce8b6d46efa083f: done: 0.067641 sec","job_status":"done","queue_duration_s":0.002465,"scheduling_latency_s":0.002506,"redis_calls":13,"redis_duration_s":0.016189,"redis_read_bytes":773,"redis_write_bytes":1111,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.007845,"redis_feature_flag_read_bytes":768,"redis_feature_flag_write_bytes":400,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00671,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001634,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.067641,"completed_at":"2026-10-03T07:04:02.200Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 282.360309] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:02.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e8cd77fcf0a2738af71fb8a0","created_at":"2026-10-03T07:04:02.142Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ff201785aee69e0d1468028ea052da7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.149Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e8cd77fcf0a2738af71fb8a0: start","job_status":"start","queue_duration_s":0.05592,"scheduling_latency_s":0.05595}1443gitlab # [ 283.137815] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"e8cd77fcf0a2738af71fb8a0","created_at":"2026-10-03T07:04:02.142Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1ff201785aee69e0d1468028ea052da7","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.149Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-e8cd77fcf0a2738af71fb8a0: done: 0.832078 sec","job_status":"done","queue_duration_s":0.05592,"scheduling_latency_s":0.05595,"redis_calls":562,"redis_duration_s":0.467835,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.466714,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.425797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.832078,"completed_at":"2026-10-03T07:04:03.037Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 283.139355] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.039Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"994cc3dbb7b3c1b027e12c50","created_at":"2026-10-03T07:04:02.166Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be43c213b9d133491e183d92bbb9f5eb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.207Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-994cc3dbb7b3c1b027e12c50: start","job_status":"start","queue_duration_s":0.83164,"scheduling_latency_s":0.831666}1445gitlab # [ 283.202345] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"994cc3dbb7b3c1b027e12c50","created_at":"2026-10-03T07:04:02.166Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be43c213b9d133491e183d92bbb9f5eb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.207Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-994cc3dbb7b3c1b027e12c50: done: 0.036231 sec","job_status":"done","queue_duration_s":0.83164,"scheduling_latency_s":0.831666,"redis_calls":13,"redis_duration_s":0.002915,"redis_read_bytes":139,"redis_write_bytes":1611,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00068,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":386,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01962,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00065984,"exclusive_lock_hold_duration_s":0.010900179999999999,"duration_s":0.036231,"completed_at":"2026-10-03T07:04:03.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 283.265399] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.077Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"544cc44e00de36b8d58ec920","created_at":"2026-10-03T07:04:02.328Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"656d719ec8599f7c17adfa3cad4ecbca","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.332Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-544cc44e00de36b8d58ec920: start","job_status":"start","queue_duration_s":0.744489,"scheduling_latency_s":0.744516}1447gitlab # [ 283.279731] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"544cc44e00de36b8d58ec920","created_at":"2026-10-03T07:04:02.328Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"656d719ec8599f7c17adfa3cad4ecbca","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.332Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-544cc44e00de36b8d58ec920: done: 0.024345 sec","job_status":"done","queue_duration_s":0.744489,"scheduling_latency_s":0.744516,"redis_calls":7,"redis_duration_s":0.0012070000000000002,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011192,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.024345,"completed_at":"2026-10-03T07:04:03.101Z","load_balancing_strategy":"primary","db_duration_s":0.008108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 283.322907] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"da785ed82e3cad34210dee1f","created_at":"2026-10-03T07:04:02.347Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e162baaefed795c4a44de4b38a230c50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.352Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-da785ed82e3cad34210dee1f: start","job_status":"start","queue_duration_s":0.75002,"scheduling_latency_s":0.75004}1449gitlab # [ 283.338242] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.110Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1450gitlab # [ 283.342282] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"da785ed82e3cad34210dee1f","created_at":"2026-10-03T07:04:02.347Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e162baaefed795c4a44de4b38a230c50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.352Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-da785ed82e3cad34210dee1f: done: 0.013511 sec","job_status":"done","queue_duration_s":0.75002,"scheduling_latency_s":0.75004,"redis_calls":7,"redis_duration_s":0.001092,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008747,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013511,"completed_at":"2026-10-03T07:04:03.116Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 283.384761] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.117Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"828b787fe11d7d58c489c33e","created_at":"2026-10-03T07:04:02.371Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e7af1217cde2deb9f1142ea9d2401e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-03T07:04:02.376Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-828b787fe11d7d58c489c33e: start","job_status":"start","queue_duration_s":0.741248,"scheduling_latency_s":0.741268}1452gitlab # [ 283.398681] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.125Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1453gitlab # [ 283.402790] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"828b787fe11d7d58c489c33e","created_at":"2026-10-03T07:04:02.371Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e7af1217cde2deb9f1142ea9d2401e8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-03T07:04:02.376Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-828b787fe11d7d58c489c33e: done: 0.013517 sec","job_status":"done","queue_duration_s":0.741248,"scheduling_latency_s":0.741268,"redis_calls":7,"redis_duration_s":0.00113,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013517,"completed_at":"2026-10-03T07:04:03.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 283.445004] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.132Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fb51a3d2fe64ff0cb2beecde","created_at":"2026-10-03T07:04:02.388Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8abcd610896b95eccdbc2e0fa5de70d0","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.394Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fb51a3d2fe64ff0cb2beecde: start","job_status":"start","queue_duration_s":0.737759,"scheduling_latency_s":0.737778}1455gitlab # [ 283.459936] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fb51a3d2fe64ff0cb2beecde","created_at":"2026-10-03T07:04:02.388Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8abcd610896b95eccdbc2e0fa5de70d0","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.394Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fb51a3d2fe64ff0cb2beecde: done: 0.017477 sec","job_status":"done","queue_duration_s":0.737759,"scheduling_latency_s":0.737778,"redis_calls":9,"redis_duration_s":0.001555,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000792,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000763,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053488,"exclusive_lock_hold_duration_s":0.00352492,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.017477,"completed_at":"2026-10-03T07:04:03.149Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001296,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 283.503626] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"d175ed7344bdb3357d79864d","created_at":"2026-10-03T07:04:02.411Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7af98607b763e9b36f4c10f7dd95261","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.417Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-d175ed7344bdb3357d79864d: start","job_status":"start","queue_duration_s":0.734162,"scheduling_latency_s":0.734182}1457gitlab # [ 283.519333] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.158Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1458gitlab # [ 283.523274] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"d175ed7344bdb3357d79864d","created_at":"2026-10-03T07:04:02.411Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7af98607b763e9b36f4c10f7dd95261","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.417Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-d175ed7344bdb3357d79864d: done: 0.012709 sec","job_status":"done","queue_duration_s":0.734162,"scheduling_latency_s":0.734182,"redis_calls":7,"redis_duration_s":0.001057,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012709,"completed_at":"2026-10-03T07:04:03.164Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 283.565675] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"91a9498a6754aa35164825b2","created_at":"2026-10-03T07:04:02.453Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63094b0f4cbbfdb2ecc890b7c206e0d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-03T07:04:02.458Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-91a9498a6754aa35164825b2: start","job_status":"start","queue_duration_s":0.70719,"scheduling_latency_s":0.70721}1460gitlab # [ 283.579378] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.187Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"91a9498a6754aa35164825b2","created_at":"2026-10-03T07:04:02.453Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"63094b0f4cbbfdb2ecc890b7c206e0d9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-03T07:04:02.458Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-91a9498a6754aa35164825b2: done: 0.022136 sec","job_status":"done","queue_duration_s":0.70719,"scheduling_latency_s":0.70721,"redis_calls":8,"redis_duration_s":0.0015639999999999999,"redis_read_bytes":133,"redis_write_bytes":811,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000148,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022136,"completed_at":"2026-10-03T07:04:03.187Z","load_balancing_strategy":"primary","db_duration_s":0.00161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 283.623209] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.189Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"43a06d86fdbd5c0bcafd1528","created_at":"2026-10-03T07:04:02.474Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"1c5a03234a4ae70a2e73620be6de95d8","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.487Z","job_size_bytes":3,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-43a06d86fdbd5c0bcafd1528: start","job_status":"start","queue_duration_s":0.701606,"scheduling_latency_s":0.701627}1462gitlab # [ 283.637874] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.207Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"43a06d86fdbd5c0bcafd1528","created_at":"2026-10-03T07:04:02.474Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"1c5a03234a4ae70a2e73620be6de95d8","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.487Z","job_size_bytes":3,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-43a06d86fdbd5c0bcafd1528: done: 0.018691 sec","job_status":"done","queue_duration_s":0.701606,"scheduling_latency_s":0.701627,"redis_calls":10,"redis_duration_s":0.0015,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000352,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026398,"duration_s":0.018691,"completed_at":"2026-10-03T07:04:03.207Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 283.681719] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2566f52ec55ed44c73b777c9","created_at":"2026-10-03T07:04:02.500Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"016c6a340e80583c9ffb64cf22877d40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.505Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2566f52ec55ed44c73b777c9: start","job_status":"start","queue_duration_s":0.70322,"scheduling_latency_s":0.703239}1464gitlab # [ 283.696749] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.222Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"2566f52ec55ed44c73b777c9","created_at":"2026-10-03T07:04:02.500Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"016c6a340e80583c9ffb64cf22877d40","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.505Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-2566f52ec55ed44c73b777c9: done: 0.013214 sec","job_status":"done","queue_duration_s":0.70322,"scheduling_latency_s":0.703239,"redis_calls":7,"redis_duration_s":0.001067,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013214,"completed_at":"2026-10-03T07:04:03.222Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 283.739494] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.223Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"26e34c2f5f6a0e285da35871","created_at":"2026-10-03T07:04:02.518Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15561a4892d70e4e15d6c59279bf05f9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.526Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-26e34c2f5f6a0e285da35871: start","job_status":"start","queue_duration_s":0.697229,"scheduling_latency_s":0.697248}1466gitlab # [ 283.754431] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"26e34c2f5f6a0e285da35871","created_at":"2026-10-03T07:04:02.518Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"15561a4892d70e4e15d6c59279bf05f9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.526Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-26e34c2f5f6a0e285da35871: done: 0.015211 sec","job_status":"done","queue_duration_s":0.697229,"scheduling_latency_s":0.697248,"redis_calls":7,"redis_duration_s":0.0011250000000000001,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000304,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015211,"completed_at":"2026-10-03T07:04:03.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 283.797638] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.239Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6e10beede00200e909607e6a","created_at":"2026-10-03T07:04:02.541Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f2d6c6a50917d6ad1b392fb93c61a1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-03T07:04:02.546Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6e10beede00200e909607e6a: start","job_status":"start","queue_duration_s":0.693345,"scheduling_latency_s":0.693365}1468gitlab # [ 283.811827] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.247Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1469gitlab # [ 283.815492] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.252Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6e10beede00200e909607e6a","created_at":"2026-10-03T07:04:02.541Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f2d6c6a50917d6ad1b392fb93c61a1d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-03T07:04:02.546Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6e10beede00200e909607e6a: done: 0.013026 sec","job_status":"done","queue_duration_s":0.693345,"scheduling_latency_s":0.693365,"redis_calls":7,"redis_duration_s":0.001056,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013026,"completed_at":"2026-10-03T07:04:03.252Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 283.857367] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c143190aa5f1dd737f505784","created_at":"2026-10-03T07:04:02.558Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e44e3f4e7b45df54a934980d395096b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.564Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c143190aa5f1dd737f505784: start","job_status":"start","queue_duration_s":0.689322,"scheduling_latency_s":0.68934}1471gitlab # [ 283.872742] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.329Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c143190aa5f1dd737f505784","created_at":"2026-10-03T07:04:02.558Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9e44e3f4e7b45df54a934980d395096b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.564Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c143190aa5f1dd737f505784: done: 0.07523 sec","job_status":"done","queue_duration_s":0.689322,"scheduling_latency_s":0.68934,"redis_calls":9,"redis_duration_s":0.0016159999999999998,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005117,"exclusive_lock_hold_duration_s":0.04425692,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.07523,"completed_at":"2026-10-03T07:04:03.329Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 283.917736] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.330Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"69b42b62ce76b3873802e3a0","created_at":"2026-10-03T07:04:02.605Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10706bc853be331aad71c29821a037af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.613Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-69b42b62ce76b3873802e3a0: start","job_status":"start","queue_duration_s":0.717643,"scheduling_latency_s":0.717664}1473gitlab # [ 283.933209] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.344Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"69b42b62ce76b3873802e3a0","created_at":"2026-10-03T07:04:02.605Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10706bc853be331aad71c29821a037af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.613Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-69b42b62ce76b3873802e3a0: done: 0.013875 sec","job_status":"done","queue_duration_s":0.717643,"scheduling_latency_s":0.717664,"redis_calls":7,"redis_duration_s":0.001158,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013875,"completed_at":"2026-10-03T07:04:03.344Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 283.975670] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b3e14e966f7ab2d40e44889a","created_at":"2026-10-03T07:04:02.630Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd517dcb6e33488adb1fd0d0ce9e9078","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.636Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b3e14e966f7ab2d40e44889a: start","job_status":"start","queue_duration_s":0.709225,"scheduling_latency_s":0.709246}1475gitlab # [ 283.991385] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b3e14e966f7ab2d40e44889a","created_at":"2026-10-03T07:04:02.630Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fd517dcb6e33488adb1fd0d0ce9e9078","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.636Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b3e14e966f7ab2d40e44889a: done: 0.044126 sec","job_status":"done","queue_duration_s":0.709225,"scheduling_latency_s":0.709246,"redis_calls":9,"redis_duration_s":0.002003,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00090406,"exclusive_lock_hold_duration_s":0.01612124,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.044126,"completed_at":"2026-10-03T07:04:03.389Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 284.037411] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.391Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"76a3e8a673cd88634701319e","created_at":"2026-10-03T07:04:02.649Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46f6790746b361f0906b8fa7be9abb15","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.655Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-76a3e8a673cd88634701319e: start","job_status":"start","queue_duration_s":0.735903,"scheduling_latency_s":0.735946}1477gitlab # [ 284.051982] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.634Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"76a3e8a673cd88634701319e","created_at":"2026-10-03T07:04:02.649Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46f6790746b361f0906b8fa7be9abb15","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.655Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-76a3e8a673cd88634701319e: done: 0.242818 sec","job_status":"done","queue_duration_s":0.735903,"scheduling_latency_s":0.735946,"redis_calls":11,"redis_duration_s":0.007408,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.196923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00068,"exclusive_lock_hold_duration_s":0.21327234,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.242818,"completed_at":"2026-10-03T07:04:03.634Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 284.098373] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.636Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c7d595554e1b9f46393386c9","created_at":"2026-10-03T07:04:02.691Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9646c5004d255370095fa1b2bbae10a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.703Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c7d595554e1b9f46393386c9: start","job_status":"start","queue_duration_s":0.932644,"scheduling_latency_s":0.93267}1479gitlab # [ 284.113551] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c7d595554e1b9f46393386c9","created_at":"2026-10-03T07:04:02.691Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9646c5004d255370095fa1b2bbae10a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.703Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c7d595554e1b9f46393386c9: done: 0.026749 sec","job_status":"done","queue_duration_s":0.932644,"scheduling_latency_s":0.93267,"redis_calls":10,"redis_duration_s":0.00221,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001275,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00782556,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.026749,"completed_at":"2026-10-03T07:04:03.662Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00466,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1480gitlab # [ 284.158339] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.664Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bc88f6c0bb69583eeb9b33b1","created_at":"2026-10-03T07:04:02.717Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbff1632877804ac43818fd1481de799","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.729Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bc88f6c0bb69583eeb9b33b1: start","job_status":"start","queue_duration_s":0.934681,"scheduling_latency_s":0.934702}1481gitlab # [ 284.172196] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"bc88f6c0bb69583eeb9b33b1","created_at":"2026-10-03T07:04:02.717Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbff1632877804ac43818fd1481de799","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.729Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-bc88f6c0bb69583eeb9b33b1: done: 0.020608 sec","job_status":"done","queue_duration_s":0.934681,"scheduling_latency_s":0.934702,"redis_calls":12,"redis_duration_s":0.002001,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000454,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00035,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028054,"duration_s":0.020608,"completed_at":"2026-10-03T07:04:03.684Z","load_balancing_strategy":"primary","db_duration_s":0.000532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 284.216397] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.686Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"713a8684dafca2c803c60435","created_at":"2026-10-03T07:04:02.747Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ee7964e89db7c408c939f77e42aeaa3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.752Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-713a8684dafca2c803c60435: start","job_status":"start","queue_duration_s":0.93382,"scheduling_latency_s":0.933841}1483gitlab # [ 284.232478] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.694Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1484gitlab # [ 284.235934] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.699Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"713a8684dafca2c803c60435","created_at":"2026-10-03T07:04:02.747Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8ee7964e89db7c408c939f77e42aeaa3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.752Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-713a8684dafca2c803c60435: done: 0.013642 sec","job_status":"done","queue_duration_s":0.93382,"scheduling_latency_s":0.933841,"redis_calls":7,"redis_duration_s":0.001099,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008378,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013642,"completed_at":"2026-10-03T07:04:03.699Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 284.279089] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"71fd8bc50472aace70e995f9","created_at":"2026-10-03T07:04:02.769Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.775Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-71fd8bc50472aace70e995f9: start","job_status":"start","queue_duration_s":0.925845,"scheduling_latency_s":0.925867}1486gitlab # [ 284.293485] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"71fd8bc50472aace70e995f9","created_at":"2026-10-03T07:04:02.769Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.775Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-71fd8bc50472aace70e995f9: done: 0.093492 sec","job_status":"done","queue_duration_s":0.925845,"scheduling_latency_s":0.925867,"redis_calls":60,"redis_duration_s":0.009875,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000546,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.008847,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000482,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08474,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.093492,"completed_at":"2026-10-03T07:04:03.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 284.336225] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.796Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"889ae67c4c8297d349957293","created_at":"2026-10-03T07:04:02.789Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"992905cdbe06cadb857e276cc069af4f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.795Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-889ae67c4c8297d349957293: start","job_status":"start","queue_duration_s":1.000544,"scheduling_latency_s":1.000567}1488gitlab # [ 284.350259] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"889ae67c4c8297d349957293","created_at":"2026-10-03T07:04:02.789Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"992905cdbe06cadb857e276cc069af4f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.795Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-889ae67c4c8297d349957293: done: 0.017437 sec","job_status":"done","queue_duration_s":1.000544,"scheduling_latency_s":1.000567,"redis_calls":9,"redis_duration_s":0.001414,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000673,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053894,"exclusive_lock_hold_duration_s":0.00369784,"duration_s":0.017437,"completed_at":"2026-10-03T07:04:03.813Z","load_balancing_strategy":"primary","db_duration_s":0.001204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 284.391805] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.814Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"0537bfbb3c57eb1ff516df4a","created_at":"2026-10-03T07:04:02.807Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8d6b1b6326c7426f2c5d3f74ebbc5b5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.820Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-0537bfbb3c57eb1ff516df4a: start","job_status":"start","queue_duration_s":0.994244,"scheduling_latency_s":0.994264}1490gitlab # [ 284.407150] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"0537bfbb3c57eb1ff516df4a","created_at":"2026-10-03T07:04:02.807Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8d6b1b6326c7426f2c5d3f74ebbc5b5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.820Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-0537bfbb3c57eb1ff516df4a: done: 0.027698 sec","job_status":"done","queue_duration_s":0.994244,"scheduling_latency_s":0.994264,"redis_calls":10,"redis_duration_s":0.001538,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000866,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000361,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002699,"duration_s":0.027698,"completed_at":"2026-10-03T07:04:03.842Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 284.452342] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.843Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7a37718c2c0111dcfbf47fb2","created_at":"2026-10-03T07:04:02.832Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.837Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7a37718c2c0111dcfbf47fb2: start","job_status":"start","queue_duration_s":1.005977,"scheduling_latency_s":1.005997}1492gitlab # [ 284.466164] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.869Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7a37718c2c0111dcfbf47fb2","created_at":"2026-10-03T07:04:02.832Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.837Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7a37718c2c0111dcfbf47fb2: done: 0.025726 sec","job_status":"done","queue_duration_s":1.005977,"scheduling_latency_s":1.005997,"redis_calls":14,"redis_duration_s":0.0021650000000000003,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000246,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001441,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025726,"completed_at":"2026-10-03T07:04:03.869Z","load_balancing_strategy":"primary","db_duration_s":0.001269,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 284.509362] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.870Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0558868d12001908d0081de8","created_at":"2026-10-03T07:04:02.856Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f0fb523fd873326a9fc7052a7267782","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.862Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0558868d12001908d0081de8: start","job_status":"start","queue_duration_s":1.00805,"scheduling_latency_s":1.00807}1494gitlab # [ 284.524247] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.878Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1495gitlab # [ 284.527508] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0558868d12001908d0081de8","created_at":"2026-10-03T07:04:02.856Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-c005b12c109044da","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f0fb523fd873326a9fc7052a7267782","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:02.862Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0558868d12001908d0081de8: done: 0.012956 sec","job_status":"done","queue_duration_s":1.00805,"scheduling_latency_s":1.00807,"redis_calls":7,"redis_duration_s":0.001112,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012956,"completed_at":"2026-10-03T07:04:03.883Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 284.570414] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e621097faceaa879b8702d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e621097faceaa879b8702d: start","job_status":"start","queue_duration_s":0.097253,"scheduling_latency_s":0.097271}1497gitlab # [ 284.583632] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.900Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04e621097faceaa879b8702d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04e621097faceaa879b8702d: done: 0.015346 sec","job_status":"done","queue_duration_s":0.097253,"scheduling_latency_s":0.097271,"redis_calls":13,"redis_duration_s":0.001972,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015346,"completed_at":"2026-10-03T07:04:03.900Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498gitlab # [ 284.624765] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b5553f6f43345feebfc4d58","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b5553f6f43345feebfc4d58: start","job_status":"start","queue_duration_s":0.113786,"scheduling_latency_s":0.113804}1499gitlab # [ 284.637672] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b5553f6f43345feebfc4d58","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b5553f6f43345feebfc4d58: done: 0.015175 sec","job_status":"done","queue_duration_s":0.113786,"scheduling_latency_s":0.113804,"redis_calls":13,"redis_duration_s":0.001922,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001278,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015175,"completed_at":"2026-10-03T07:04:03.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 284.679438] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"756c97b927a57024da94013c","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-756c97b927a57024da94013c: start","job_status":"start","queue_duration_s":0.130157,"scheduling_latency_s":0.130176}1501gitlab # [ 284.692691] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.932Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"756c97b927a57024da94013c","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-756c97b927a57024da94013c: done: 0.015119 sec","job_status":"done","queue_duration_s":0.130157,"scheduling_latency_s":0.130176,"redis_calls":13,"redis_duration_s":0.001689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015119,"completed_at":"2026-10-03T07:04:03.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 284.733756] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79b983560dbbe8c5453f72dd","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79b983560dbbe8c5453f72dd: start","job_status":"start","queue_duration_s":0.146448,"scheduling_latency_s":0.146467}1503gitlab # [ 284.747069] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79b983560dbbe8c5453f72dd","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79b983560dbbe8c5453f72dd: done: 0.014968 sec","job_status":"done","queue_duration_s":0.146448,"scheduling_latency_s":0.146467,"redis_calls":13,"redis_duration_s":0.00165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010172,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014968,"completed_at":"2026-10-03T07:04:03.949Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504gitlab # [ 284.789551] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.950Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7e792666802d6ccd7c313ed","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7e792666802d6ccd7c313ed: start","job_status":"start","queue_duration_s":0.162591,"scheduling_latency_s":0.162611}1505gitlab # [ 284.804538] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7e792666802d6ccd7c313ed","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7e792666802d6ccd7c313ed: done: 0.014585 sec","job_status":"done","queue_duration_s":0.162591,"scheduling_latency_s":0.162611,"redis_calls":13,"redis_duration_s":0.0016480000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009949,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014585,"completed_at":"2026-10-03T07:04:03.964Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506gitlab # [ 284.869147] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3de3e57a512d33a7b656a9b5","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3de3e57a512d33a7b656a9b5: start","job_status":"start","queue_duration_s":0.178352,"scheduling_latency_s":0.178371}1507gitlab # [ 284.887017] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3de3e57a512d33a7b656a9b5","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3de3e57a512d33a7b656a9b5: done: 0.015034 sec","job_status":"done","queue_duration_s":0.178352,"scheduling_latency_s":0.178371,"redis_calls":13,"redis_duration_s":0.0018449999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015034,"completed_at":"2026-10-03T07:04:03.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508gitlab # [ 284.930935] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"499d1ff460cb8fb670be44d7","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-499d1ff460cb8fb670be44d7: start","job_status":"start","queue_duration_s":0.19456,"scheduling_latency_s":0.19458}1509gitlab # [ 284.943981] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"499d1ff460cb8fb670be44d7","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-499d1ff460cb8fb670be44d7: done: 0.014737 sec","job_status":"done","queue_duration_s":0.19456,"scheduling_latency_s":0.19458,"redis_calls":13,"redis_duration_s":0.001655,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014737,"completed_at":"2026-10-03T07:04:03.996Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510gitlab # [ 284.984962] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:03.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7433d0421d1fd7323811ace6","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7433d0421d1fd7323811ace6: start","job_status":"start","queue_duration_s":0.210454,"scheduling_latency_s":0.210472}1511gitlab # [ 284.997808] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7433d0421d1fd7323811ace6","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7433d0421d1fd7323811ace6: done: 0.014519 sec","job_status":"done","queue_duration_s":0.210454,"scheduling_latency_s":0.210472,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000609,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014519,"completed_at":"2026-10-03T07:04:04.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512gitlab # [ 285.038794] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a765455e3f44ddfc14c0a9a2","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a765455e3f44ddfc14c0a9a2: start","job_status":"start","queue_duration_s":0.226124,"scheduling_latency_s":0.226143}1513gitlab # [ 285.052747] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a765455e3f44ddfc14c0a9a2","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a765455e3f44ddfc14c0a9a2: done: 0.014721 sec","job_status":"done","queue_duration_s":0.226124,"scheduling_latency_s":0.226143,"redis_calls":13,"redis_duration_s":0.0017000000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014721,"completed_at":"2026-10-03T07:04:04.028Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 285.093585] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d45fcdd204e3cb414a6421b","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d45fcdd204e3cb414a6421b: start","job_status":"start","queue_duration_s":0.242063,"scheduling_latency_s":0.242083}1515gitlab # [ 285.106971] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d45fcdd204e3cb414a6421b","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d45fcdd204e3cb414a6421b: done: 0.014596 sec","job_status":"done","queue_duration_s":0.242063,"scheduling_latency_s":0.242083,"redis_calls":13,"redis_duration_s":0.001618,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014596,"completed_at":"2026-10-03T07:04:04.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516gitlab # [ 285.149309] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a300fb19f19c1e4732f695d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a300fb19f19c1e4732f695d: start","job_status":"start","queue_duration_s":0.257851,"scheduling_latency_s":0.257871}1517gitlab # [ 285.162609] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a300fb19f19c1e4732f695d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a300fb19f19c1e4732f695d: done: 0.014609 sec","job_status":"done","queue_duration_s":0.257851,"scheduling_latency_s":0.257871,"redis_calls":13,"redis_duration_s":0.001641,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014609,"completed_at":"2026-10-03T07:04:04.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518gitlab # [ 285.204215] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e24ce1d99b0867e134ca47d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e24ce1d99b0867e134ca47d: start","job_status":"start","queue_duration_s":0.273616,"scheduling_latency_s":0.273636}1519gitlab # [ 285.217069] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.076Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e24ce1d99b0867e134ca47d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e24ce1d99b0867e134ca47d: done: 0.014969 sec","job_status":"done","queue_duration_s":0.273616,"scheduling_latency_s":0.273636,"redis_calls":13,"redis_duration_s":0.001643,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014969,"completed_at":"2026-10-03T07:04:04.076Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 285.258491] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee9d1436523b1748ef228e3f","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee9d1436523b1748ef228e3f: start","job_status":"start","queue_duration_s":0.289846,"scheduling_latency_s":0.289866}1521gitlab # [ 285.271540] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee9d1436523b1748ef228e3f","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee9d1436523b1748ef228e3f: done: 0.014655 sec","job_status":"done","queue_duration_s":0.289846,"scheduling_latency_s":0.289866,"redis_calls":13,"redis_duration_s":0.001647,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014655,"completed_at":"2026-10-03T07:04:04.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 285.312734] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a50821ebc2e8da586c59059","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a50821ebc2e8da586c59059: start","job_status":"start","queue_duration_s":0.30569,"scheduling_latency_s":0.305709}1523gitlab # [ 285.325612] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.108Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4a50821ebc2e8da586c59059","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4a50821ebc2e8da586c59059: done: 0.014644 sec","job_status":"done","queue_duration_s":0.30569,"scheduling_latency_s":0.305709,"redis_calls":13,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014644,"completed_at":"2026-10-03T07:04:04.107Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 285.366121] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59cfb2a8fec904bdc7739b88","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59cfb2a8fec904bdc7739b88: start","job_status":"start","queue_duration_s":0.321508,"scheduling_latency_s":0.321527}1525gitlab # [ 285.378932] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"59cfb2a8fec904bdc7739b88","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-59cfb2a8fec904bdc7739b88: done: 0.014481 sec","job_status":"done","queue_duration_s":0.321508,"scheduling_latency_s":0.321527,"redis_calls":13,"redis_duration_s":0.00164,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001055,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014481,"completed_at":"2026-10-03T07:04:04.123Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 285.419701] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4f5f6cf9bcf2a150eb38959","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4f5f6cf9bcf2a150eb38959: start","job_status":"start","queue_duration_s":0.337205,"scheduling_latency_s":0.337224}1527gitlab # [ 285.432800] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4f5f6cf9bcf2a150eb38959","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4f5f6cf9bcf2a150eb38959: done: 0.015083 sec","job_status":"done","queue_duration_s":0.337205,"scheduling_latency_s":0.337224,"redis_calls":13,"redis_duration_s":0.001658,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000578,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015083,"completed_at":"2026-10-03T07:04:04.139Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 285.474283] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eb569f8afad1d511236af5e","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eb569f8afad1d511236af5e: start","job_status":"start","queue_duration_s":0.353497,"scheduling_latency_s":0.353517}1529gitlab # [ 285.487179] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7eb569f8afad1d511236af5e","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7eb569f8afad1d511236af5e: done: 0.014712 sec","job_status":"done","queue_duration_s":0.353497,"scheduling_latency_s":0.353517,"redis_calls":13,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014712,"completed_at":"2026-10-03T07:04:04.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 285.528032] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d0a5c8c01dec45fc0de3a23","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d0a5c8c01dec45fc0de3a23: start","job_status":"start","queue_duration_s":0.369426,"scheduling_latency_s":0.369445}1531gitlab # [ 285.540798] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d0a5c8c01dec45fc0de3a23","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d0a5c8c01dec45fc0de3a23: done: 0.014468 sec","job_status":"done","queue_duration_s":0.369426,"scheduling_latency_s":0.369445,"redis_calls":13,"redis_duration_s":0.0016679999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014468,"completed_at":"2026-10-03T07:04:04.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532gitlab # [ 285.581511] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75d846671d257fd7e9882cea","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75d846671d257fd7e9882cea: start","job_status":"start","queue_duration_s":0.38511,"scheduling_latency_s":0.385129}1533gitlab # [ 285.594241] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75d846671d257fd7e9882cea","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75d846671d257fd7e9882cea: done: 0.015171 sec","job_status":"done","queue_duration_s":0.38511,"scheduling_latency_s":0.385129,"redis_calls":13,"redis_duration_s":0.001708,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015171,"completed_at":"2026-10-03T07:04:04.187Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534gitlab # [ 285.634843] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20f3891be18f931136a18d5a","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20f3891be18f931136a18d5a: start","job_status":"start","queue_duration_s":0.401462,"scheduling_latency_s":0.401483}1535gitlab # [ 285.647897] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20f3891be18f931136a18d5a","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20f3891be18f931136a18d5a: done: 0.014586 sec","job_status":"done","queue_duration_s":0.401462,"scheduling_latency_s":0.401483,"redis_calls":13,"redis_duration_s":0.001663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014586,"completed_at":"2026-10-03T07:04:04.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536gitlab # [ 285.689314] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93b0caa15dcde90114668707","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93b0caa15dcde90114668707: start","job_status":"start","queue_duration_s":0.417297,"scheduling_latency_s":0.417318}1537gitlab # [ 285.702062] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93b0caa15dcde90114668707","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93b0caa15dcde90114668707: done: 0.014382 sec","job_status":"done","queue_duration_s":0.417297,"scheduling_latency_s":0.417318,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014382,"completed_at":"2026-10-03T07:04:04.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538gitlab # [ 285.742805] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65b451e84fe1a441956c86a2","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65b451e84fe1a441956c86a2: start","job_status":"start","queue_duration_s":0.432914,"scheduling_latency_s":0.432934}1539gitlab # [ 285.755811] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65b451e84fe1a441956c86a2","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65b451e84fe1a441956c86a2: done: 0.014696 sec","job_status":"done","queue_duration_s":0.432914,"scheduling_latency_s":0.432934,"redis_calls":13,"redis_duration_s":0.001701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000603,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014696,"completed_at":"2026-10-03T07:04:04.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540gitlab # [ 285.796592] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"264c193380d6f0451d75c1b1","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-264c193380d6f0451d75c1b1: start","job_status":"start","queue_duration_s":0.448788,"scheduling_latency_s":0.448807}1541gitlab # [ 285.809480] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"264c193380d6f0451d75c1b1","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-264c193380d6f0451d75c1b1: done: 0.014573 sec","job_status":"done","queue_duration_s":0.448788,"scheduling_latency_s":0.448807,"redis_calls":13,"redis_duration_s":0.001683,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014573,"completed_at":"2026-10-03T07:04:04.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1542gitlab # [ 285.850186] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b52ee422caead55547c12758","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b52ee422caead55547c12758: start","job_status":"start","queue_duration_s":0.46454,"scheduling_latency_s":0.464559}1543gitlab # [ 285.862926] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b52ee422caead55547c12758","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b52ee422caead55547c12758: done: 0.014544 sec","job_status":"done","queue_duration_s":0.46454,"scheduling_latency_s":0.464559,"redis_calls":13,"redis_duration_s":0.001714,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014544,"completed_at":"2026-10-03T07:04:04.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544gitlab # [ 285.905239] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"639677ec6d7538c0660bd7b7","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-639677ec6d7538c0660bd7b7: start","job_status":"start","queue_duration_s":0.480229,"scheduling_latency_s":0.480248}1545gitlab # [ 285.918024] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"639677ec6d7538c0660bd7b7","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-639677ec6d7538c0660bd7b7: done: 0.014816 sec","job_status":"done","queue_duration_s":0.480229,"scheduling_latency_s":0.480248,"redis_calls":13,"redis_duration_s":0.001829,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010129,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014816,"completed_at":"2026-10-03T07:04:04.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546gitlab # [ 285.958829] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2297aeccfe15e8ee3532cf46","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2297aeccfe15e8ee3532cf46: start","job_status":"start","queue_duration_s":0.496194,"scheduling_latency_s":0.496215}1547gitlab # [ 285.972164] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2297aeccfe15e8ee3532cf46","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2297aeccfe15e8ee3532cf46: done: 0.014721 sec","job_status":"done","queue_duration_s":0.496194,"scheduling_latency_s":0.496215,"redis_calls":13,"redis_duration_s":0.001653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014721,"completed_at":"2026-10-03T07:04:04.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1548gitlab # [ 286.012826] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.299Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1636622ba6e1ab8022d2bc0","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1636622ba6e1ab8022d2bc0: start","job_status":"start","queue_duration_s":0.512085,"scheduling_latency_s":0.512106}1549gitlab # [ 286.025540] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.314Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1636622ba6e1ab8022d2bc0","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1636622ba6e1ab8022d2bc0: done: 0.014612 sec","job_status":"done","queue_duration_s":0.512085,"scheduling_latency_s":0.512106,"redis_calls":13,"redis_duration_s":0.0017139999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014612,"completed_at":"2026-10-03T07:04:04.314Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1550gitlab # [ 286.066254] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5f0ea9f8b95611670b777dd","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5f0ea9f8b95611670b777dd: start","job_status":"start","queue_duration_s":0.527879,"scheduling_latency_s":0.527899}1551gitlab # [ 286.080212] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.330Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f5f0ea9f8b95611670b777dd","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f5f0ea9f8b95611670b777dd: done: 0.014659 sec","job_status":"done","queue_duration_s":0.527879,"scheduling_latency_s":0.527899,"redis_calls":13,"redis_duration_s":0.0016820000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014659,"completed_at":"2026-10-03T07:04:04.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552gitlab # [ 286.121286] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9cf5c67f2084ea369acca3a","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9cf5c67f2084ea369acca3a: start","job_status":"start","queue_duration_s":0.544191,"scheduling_latency_s":0.544212}1553gitlab # [ 286.134153] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9cf5c67f2084ea369acca3a","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9cf5c67f2084ea369acca3a: done: 0.015189 sec","job_status":"done","queue_duration_s":0.544191,"scheduling_latency_s":0.544212,"redis_calls":13,"redis_duration_s":0.00172,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015189,"completed_at":"2026-10-03T07:04:04.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1554gitlab # [ 286.175067] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"360fcaa605c688859a1d682d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-360fcaa605c688859a1d682d: start","job_status":"start","queue_duration_s":0.560573,"scheduling_latency_s":0.560592}1555gitlab # [ 286.188234] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"360fcaa605c688859a1d682d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-360fcaa605c688859a1d682d: done: 0.014498 sec","job_status":"done","queue_duration_s":0.560573,"scheduling_latency_s":0.560592,"redis_calls":13,"redis_duration_s":0.001629,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014498,"completed_at":"2026-10-03T07:04:04.362Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1556gitlab # [ 286.228714] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4288460eb808b324fb656cbc","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4288460eb808b324fb656cbc: start","job_status":"start","queue_duration_s":0.576289,"scheduling_latency_s":0.576309}1557gitlab # [ 286.241471] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4288460eb808b324fb656cbc","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4288460eb808b324fb656cbc: done: 0.014922 sec","job_status":"done","queue_duration_s":0.576289,"scheduling_latency_s":0.576309,"redis_calls":13,"redis_duration_s":0.001676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014922,"completed_at":"2026-10-03T07:04:04.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1558gitlab # [ 286.282467] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b04bd16ec4bd36c274cda70e","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b04bd16ec4bd36c274cda70e: start","job_status":"start","queue_duration_s":0.59248,"scheduling_latency_s":0.5925}1559gitlab # [ 286.295550] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.394Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b04bd16ec4bd36c274cda70e","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b04bd16ec4bd36c274cda70e: done: 0.014828 sec","job_status":"done","queue_duration_s":0.59248,"scheduling_latency_s":0.5925,"redis_calls":13,"redis_duration_s":0.001692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014828,"completed_at":"2026-10-03T07:04:04.394Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1560gitlab # [ 286.336444] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71831993fe947110d855bfc1","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71831993fe947110d855bfc1: start","job_status":"start","queue_duration_s":0.608524,"scheduling_latency_s":0.608543}1561gitlab # [ 286.349132] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71831993fe947110d855bfc1","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71831993fe947110d855bfc1: done: 0.014447 sec","job_status":"done","queue_duration_s":0.608524,"scheduling_latency_s":0.608543,"redis_calls":13,"redis_duration_s":0.001653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000591,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014447,"completed_at":"2026-10-03T07:04:04.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1562gitlab # [ 286.389514] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e53db539f6fc0a763a28f0","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e53db539f6fc0a763a28f0: start","job_status":"start","queue_duration_s":0.624156,"scheduling_latency_s":0.624175}1563gitlab # [ 286.402387] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e53db539f6fc0a763a28f0","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e53db539f6fc0a763a28f0: done: 0.014642 sec","job_status":"done","queue_duration_s":0.624156,"scheduling_latency_s":0.624175,"redis_calls":13,"redis_duration_s":0.0016939999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014642,"completed_at":"2026-10-03T07:04:04.426Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1564gitlab # [ 286.443186] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20c8addc42ecf490330b9613","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20c8addc42ecf490330b9613: start","job_status":"start","queue_duration_s":0.639934,"scheduling_latency_s":0.639953}1565gitlab # [ 286.456133] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20c8addc42ecf490330b9613","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20c8addc42ecf490330b9613: done: 0.014717 sec","job_status":"done","queue_duration_s":0.639934,"scheduling_latency_s":0.639953,"redis_calls":13,"redis_duration_s":0.0016610000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014717,"completed_at":"2026-10-03T07:04:04.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1566gitlab # [ 286.497360] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"387e3b9ce41f64a7390ab17a","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-387e3b9ce41f64a7390ab17a: start","job_status":"start","queue_duration_s":0.655937,"scheduling_latency_s":0.655957}1567gitlab # [ 286.510275] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.458Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"387e3b9ce41f64a7390ab17a","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-387e3b9ce41f64a7390ab17a: done: 0.014611 sec","job_status":"done","queue_duration_s":0.655937,"scheduling_latency_s":0.655957,"redis_calls":13,"redis_duration_s":0.0017009999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001081,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014611,"completed_at":"2026-10-03T07:04:04.458Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1568gitlab # [ 286.551056] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.459Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a19e4157c78a47ac97c8b691","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a19e4157c78a47ac97c8b691: start","job_status":"start","queue_duration_s":0.671762,"scheduling_latency_s":0.671781}1569gitlab # [ 286.564185] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a19e4157c78a47ac97c8b691","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a19e4157c78a47ac97c8b691: done: 0.01484 sec","job_status":"done","queue_duration_s":0.671762,"scheduling_latency_s":0.671781,"redis_calls":13,"redis_duration_s":0.001709,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01484,"completed_at":"2026-10-03T07:04:04.474Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1570gitlab # [ 286.604954] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bcda039ebf6f7950a35175c","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bcda039ebf6f7950a35175c: start","job_status":"start","queue_duration_s":0.687784,"scheduling_latency_s":0.687804}1571gitlab # [ 286.617734] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bcda039ebf6f7950a35175c","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bcda039ebf6f7950a35175c: done: 0.015019 sec","job_status":"done","queue_duration_s":0.687784,"scheduling_latency_s":0.687804,"redis_calls":13,"redis_duration_s":0.001748,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015019,"completed_at":"2026-10-03T07:04:04.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1572gitlab # [ 286.658427] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a30e8a21977d2bd0f7438513","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a30e8a21977d2bd0f7438513: start","job_status":"start","queue_duration_s":0.703958,"scheduling_latency_s":0.703977}1573gitlab # [ 286.671471] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a30e8a21977d2bd0f7438513","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a30e8a21977d2bd0f7438513: done: 0.014617 sec","job_status":"done","queue_duration_s":0.703958,"scheduling_latency_s":0.703977,"redis_calls":13,"redis_duration_s":0.001716,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014617,"completed_at":"2026-10-03T07:04:04.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1574gitlab # [ 286.713069] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c70cd176fb06fbb1b0529c7","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c70cd176fb06fbb1b0529c7: start","job_status":"start","queue_duration_s":0.71977,"scheduling_latency_s":0.719789}1575gitlab # [ 286.725950] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c70cd176fb06fbb1b0529c7","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c70cd176fb06fbb1b0529c7: done: 0.014711 sec","job_status":"done","queue_duration_s":0.71977,"scheduling_latency_s":0.719789,"redis_calls":13,"redis_duration_s":0.001665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000593,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014711,"completed_at":"2026-10-03T07:04:04.522Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1576gitlab # [ 286.766348] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8329daa314dc37ab151eeb33","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8329daa314dc37ab151eeb33: start","job_status":"start","queue_duration_s":0.735697,"scheduling_latency_s":0.735716}1577gitlab # [ 286.779812] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8329daa314dc37ab151eeb33","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8329daa314dc37ab151eeb33: done: 0.014677 sec","job_status":"done","queue_duration_s":0.735697,"scheduling_latency_s":0.735716,"redis_calls":13,"redis_duration_s":0.001676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014677,"completed_at":"2026-10-03T07:04:04.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1578gitlab # [ 286.821228] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"029b15ac28ebe88c2d734335","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-029b15ac28ebe88c2d734335: start","job_status":"start","queue_duration_s":0.75156,"scheduling_latency_s":0.751579}1579gitlab # [ 286.834086] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"029b15ac28ebe88c2d734335","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-029b15ac28ebe88c2d734335: done: 0.014713 sec","job_status":"done","queue_duration_s":0.75156,"scheduling_latency_s":0.751579,"redis_calls":13,"redis_duration_s":0.0016420000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014713,"completed_at":"2026-10-03T07:04:04.553Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1580gitlab # [ 286.875162] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"164b1893c8b5e6eccbf9be8f","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-164b1893c8b5e6eccbf9be8f: start","job_status":"start","queue_duration_s":0.767492,"scheduling_latency_s":0.767511}1581gitlab # [ 286.889109] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"164b1893c8b5e6eccbf9be8f","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-164b1893c8b5e6eccbf9be8f: done: 0.014784 sec","job_status":"done","queue_duration_s":0.767492,"scheduling_latency_s":0.767511,"redis_calls":13,"redis_duration_s":0.0016910000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000612,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014784,"completed_at":"2026-10-03T07:04:04.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1582gitlab # [ 286.929931] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.571Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e87d0708712da9b9f4c25e62","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e87d0708712da9b9f4c25e62: start","job_status":"start","queue_duration_s":0.783524,"scheduling_latency_s":0.783543}1583gitlab # [ 286.942908] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e87d0708712da9b9f4c25e62","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e87d0708712da9b9f4c25e62: done: 0.01495 sec","job_status":"done","queue_duration_s":0.783524,"scheduling_latency_s":0.783543,"redis_calls":13,"redis_duration_s":0.001888,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01495,"completed_at":"2026-10-03T07:04:04.586Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1584gitlab # [ 286.983780] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.587Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b823f66770fdddf488c86b1","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b823f66770fdddf488c86b1: start","job_status":"start","queue_duration_s":0.799632,"scheduling_latency_s":0.799652}1585gitlab # [ 286.996885] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.602Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b823f66770fdddf488c86b1","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b823f66770fdddf488c86b1: done: 0.014765 sec","job_status":"done","queue_duration_s":0.799632,"scheduling_latency_s":0.799652,"redis_calls":13,"redis_duration_s":0.001712,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000607,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001105,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014765,"completed_at":"2026-10-03T07:04:04.602Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1586gitlab # [ 287.037483] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.603Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4752aed9d0604e609274132f","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4752aed9d0604e609274132f: start","job_status":"start","queue_duration_s":0.815612,"scheduling_latency_s":0.815631}1587gitlab # [ 287.050342] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4752aed9d0604e609274132f","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4752aed9d0604e609274132f: done: 0.014665 sec","job_status":"done","queue_duration_s":0.815612,"scheduling_latency_s":0.815631,"redis_calls":13,"redis_duration_s":0.0017289999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014665,"completed_at":"2026-10-03T07:04:04.617Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1588gitlab # [ 287.091265] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"294a7271ecc24dfc5e986e49","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-294a7271ecc24dfc5e986e49: start","job_status":"start","queue_duration_s":0.83157,"scheduling_latency_s":0.83159}1589gitlab # [ 287.105249] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"294a7271ecc24dfc5e986e49","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-294a7271ecc24dfc5e986e49: done: 0.059078 sec","job_status":"done","queue_duration_s":0.83157,"scheduling_latency_s":0.83159,"redis_calls":13,"redis_duration_s":0.0017820000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.059078,"completed_at":"2026-10-03T07:04:04.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1590gitlab # [ 287.145856] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e5cc2d577ffe49ccb4b73e","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e5cc2d577ffe49ccb4b73e: start","job_status":"start","queue_duration_s":0.891802,"scheduling_latency_s":0.89182}1591gitlab # [ 287.158656] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e5cc2d577ffe49ccb4b73e","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e5cc2d577ffe49ccb4b73e: done: 0.016492 sec","job_status":"done","queue_duration_s":0.891802,"scheduling_latency_s":0.89182,"redis_calls":13,"redis_duration_s":0.00183,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016492,"completed_at":"2026-10-03T07:04:04.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1592gitlab # [ 287.199698] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf0569667f4547f075fdaae9","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf0569667f4547f075fdaae9: start","job_status":"start","queue_duration_s":0.910123,"scheduling_latency_s":0.910143}1593gitlab # [ 287.212850] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf0569667f4547f075fdaae9","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf0569667f4547f075fdaae9: done: 0.017141 sec","job_status":"done","queue_duration_s":0.910123,"scheduling_latency_s":0.910143,"redis_calls":13,"redis_duration_s":0.002018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001283,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017141,"completed_at":"2026-10-03T07:04:04.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1594gitlab # [ 287.253308] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c75e05feccdad1e46adc61d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c75e05feccdad1e46adc61d: start","job_status":"start","queue_duration_s":0.928536,"scheduling_latency_s":0.928555}1595gitlab # [ 287.266411] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c75e05feccdad1e46adc61d","created_at":"2026-10-03T07:04:03.710Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b6a62a9a7c604639","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6c1378595ea2d4ea5f05c30172b2b45b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.787Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c75e05feccdad1e46adc61d: done: 0.015307 sec","job_status":"done","queue_duration_s":0.928536,"scheduling_latency_s":0.928555,"redis_calls":13,"redis_duration_s":0.0015669999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015307,"completed_at":"2026-10-03T07:04:04.731Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1596gitlab # [ 287.308097] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.732Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"985805218f881a107c056a59","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-985805218f881a107c056a59: start","job_status":"start","queue_duration_s":0.868819,"scheduling_latency_s":0.868839}1597gitlab # [ 287.320752] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.751Z","message":"No existing merge request to be cleaned up."}1598gitlab # [ 287.322482] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.761Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"985805218f881a107c056a59","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-985805218f881a107c056a59: done: 0.029071 sec","job_status":"done","queue_duration_s":0.868819,"scheduling_latency_s":0.868839,"redis_calls":10,"redis_duration_s":0.001812,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016326,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029071,"completed_at":"2026-10-03T07:04:04.761Z","load_balancing_strategy":"primary","db_duration_s":0.012953,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1599gitlab # [ 287.362946] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.763Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"68f756d85018f09193bc4564","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-68f756d85018f09193bc4564: start","job_status":"start","queue_duration_s":0.899177,"scheduling_latency_s":0.899198}1600gitlab # [ 287.376051] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.775Z","message":"No existing merge request to be cleaned up."}1601gitlab # [ 287.377755] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.785Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"68f756d85018f09193bc4564","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-68f756d85018f09193bc4564: done: 0.022456 sec","job_status":"done","queue_duration_s":0.899177,"scheduling_latency_s":0.899198,"redis_calls":10,"redis_duration_s":0.002123,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001019,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001104,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01561,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022456,"completed_at":"2026-10-03T07:04:04.785Z","load_balancing_strategy":"primary","db_duration_s":0.001831,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1602gitlab # [ 287.417781] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.786Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"68762a3daad5a14a5c757cb7","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-68762a3daad5a14a5c757cb7: start","job_status":"start","queue_duration_s":0.922918,"scheduling_latency_s":0.922937}1603gitlab # [ 287.430458] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.798Z","message":"No existing merge request to be cleaned up."}1604gitlab # [ 287.432183] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.806Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"68762a3daad5a14a5c757cb7","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-68762a3daad5a14a5c757cb7: done: 0.019887 sec","job_status":"done","queue_duration_s":0.922918,"scheduling_latency_s":0.922937,"redis_calls":10,"redis_duration_s":0.001987,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001172,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019887,"completed_at":"2026-10-03T07:04:04.806Z","load_balancing_strategy":"primary","db_duration_s":0.001909,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1605gitlab # [ 287.471944] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.808Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0d0d945a0dadd2462bc86518","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0d0d945a0dadd2462bc86518: start","job_status":"start","queue_duration_s":0.945004,"scheduling_latency_s":0.945023}1606gitlab # [ 287.484957] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.819Z","message":"No existing merge request to be cleaned up."}1607gitlab # [ 287.486668] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:04.829Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0d0d945a0dadd2462bc86518","created_at":"2026-10-03T07:04:03.854Z","trace_propagation_headers":{"sentry-trace":"5151f1e38d334d93a93df9e652b499f1-b0999036002c42bd","baggage":"sentry-trace_id=5151f1e38d334d93a93df9e652b499f1,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"e4a1b60339b752098084ff7b091ffa89","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:03.863Z","job_size_bytes":2,"pid":1946,"sidekiq_tid":"mar6","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0d0d945a0dadd2462bc86518: done: 0.020252 sec","job_status":"done","queue_duration_s":0.945004,"scheduling_latency_s":0.945023,"redis_calls":10,"redis_duration_s":0.002013,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001193,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020252,"completed_at":"2026-10-03T07:04:04.829Z","load_balancing_strategy":"primary","db_duration_s":0.001691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1608gitlab # [ 308.197172] redis[796]: 10 changes in 300 seconds. Saving...1609gitlab # [ 308.201922] redis[796]: Background saving started by pid 33501610gitlab # [ 308.204455] redis[3350]: BGSAVE done, 453 keys saved, 0 keys skipped, 117403 bytes written.1611gitlab # [ 308.206781] redis[3350]: DB saved on disk1612gitlab # [ 308.208094] redis[3350]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1613gitlab # [ 308.298077] redis[796]: Background saving terminated with success1614gitlab # [ 309.773940] postgres[861]: [861] LOG: checkpoint starting: time1615gitlab # [ 311.472951] gitaly[934]: time="2026-10-03T07:04:31.372Z" level=info msg="burden monitor stats" pid=9341616gitlab # [ 312.991816] postgres[3357]: [3357] ERROR: relation "table_without_models" does not exist1617gitlab # [ 312.992089] postgres[3357]: [3357] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1618gitlab # [ 318.618925] runuser[2990]: pam_unix(runuser:session): session closed for user gitlab1619gitlab # [ 318.620000] setup-gitlab-start[2976]: + systemctl restart gitlab-sidekiq.service1620gitlab # [ 318.646715] systemd[1]: Stopping gitlab-sidekiq.service...1621gitlab # [ 318.648545] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:38.546Z","message":"Shutting down"}1622gitlab # [ 318.650094] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:38.547Z","message":"Scheduler exiting..."}1623gitlab # [ 318.651630] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:38.547Z","message":"Terminating quiet threads for default capsule"}1624gitlab # [ 318.653566] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:38.548Z","message":"Scheduler exiting..."}1625gitlab # [ 319.148983] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:39.049Z","message":"Pausing to allow jobs to finish..."}1626gitlab # [ 320.652326] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:40.551Z","message":"Bye!"}1627gitlab # [ 320.678249] sidekiq-cluster[1946]: {"severity":"INFO","time":"2026-10-03T07:04:40.578Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1946,"worker_id":"sidekiq_0","memwd_rss_bytes":936382464,"retry":0}1628gitlab # [ 321.745423] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1629gitlab # [ 321.745678] systemd[1]: Stopped gitlab-sidekiq.service.1630gitlab # [ 321.745848] systemd[1]: gitlab-sidekiq.service: Consumed 2min 937ms CPU time over 3min 34.129s wall clock time, 942.9M memory peak, 4.5M read from disk.1631gitlab # [ 321.753376] systemd[1]: Started gitlab-sidekiq.service.1632gitlab # [ 321.756549] setup-gitlab-start[2976]: + mkdir -p /var/lib/secrets1633gitlab # [ 321.760629] setup-gitlab-start[2976]: + echo nixbot-test-token-of-20-chars1634gitlab # [ 321.762263] setup-gitlab-start[2976]: + chmod 644 /var/lib/secrets/gitlab-token1635gitlab # [ 321.764277] setup-gitlab-start[2976]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1636gitlab # [ 322.244845] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:42 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 4521637gitlab # [ 323.012827] gitaly[934]: time="2026-10-03T07:04:42.912Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:52.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:42.911" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1638gitlab # [ 323.016305] gitaly[934]: time="2026-10-03T07:04:42.912Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:52.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:42.911" grpc.time_ms=0.636 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1639gitlab # [ 323.207212] gitaly[934]: time="2026-10-03T07:04:43.107Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.107" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1640gitlab # [ 323.242029] gitaly[934]: time="2026-10-03T07:04:43.107Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.107" grpc.time_ms=0.412 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1641gitlab # [ 323.294847] gitaly[934]: time="2026-10-03T07:04:43.195Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY diskcache=19a42fb9-f0a0-4cea-8ae0-3ed586b00b97 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-03T07:04:53.050" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.150" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1642gitlab # [ 323.312291] gitaly[934]: time="2026-10-03T07:04:43.195Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=23 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1441 command.oublock=32 command.real_time_ms=27 command.spawn_token_fork_ms=1 command.system_time_ms=19 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-03T07:04:53.050" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.150" grpc.time_ms=44.71 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1643gitlab # [ 323.333757] gitaly[934]: time="2026-10-03T07:04:43.209Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.108" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.209" pid=934 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1644gitlab # [ 323.348131] gitaly[934]: time="2026-10-03T07:04:43.209Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.108" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.209" grpc.time_ms=0.855 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1645gitlab # [ 323.363137] gitaly[934]: time="2026-10-03T07:04:43.227Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:04:53.114" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.215" grpc.time_ms=11.312 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1646gitlab # [ 323.412078] gitaly[934]: time="2026-10-03T07:04:43.312Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.211" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.312" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1647gitlab # [ 323.428883] gitaly[934]: time="2026-10-03T07:04:43.312Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.211" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.312" grpc.time_ms=0.383 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1648gitlab # [ 323.449433] gitaly[934]: time="2026-10-03T07:04:43.349Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY diskcache=0cc76fd5-f9a3-4732-8081-6a2dc863e416 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-03T07:04:53.214" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.315" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1649gitlab # [ 323.466011] gitaly[934]: time="2026-10-03T07:04:43.350Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=32 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-03T07:04:53.214" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.315" grpc.time_ms=34.371 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1650gitlab # [ 323.484473] gitaly[934]: time="2026-10-03T07:04:43.357Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.255" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.356" pid=934 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1651gitlab # [ 323.497130] gitaly[934]: time="2026-10-03T07:04:43.357Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.255" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.356" grpc.time_ms=0.983 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1652gitlab # [ 323.510068] gitaly[934]: time="2026-10-03T07:04:43.369Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:04:53.260" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.361" grpc.time_ms=8.141 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1653gitlab # [ 323.526102] gitaly[934]: time="2026-10-03T07:04:43.400Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:04:53.299" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.400" grpc.time_ms=0.238 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1654gitlab # [ 323.537977] gitaly[934]: time="2026-10-03T07:04:43.408Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-03T07:04:53.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:43.407" grpc.time_ms=0.388 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1655gitlab # [ 324.228533] gitaly[934]: time="2026-10-03T07:04:44.128Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4098KXNS0N7VPMTNG97FBGY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:04:54.016" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:44.117" grpc.time_ms=11.064 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1656gitlab # [ 324.849422] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:44 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 25831657gitlab # [ 324.852498] setup-gitlab-start[3429]: {"id":1,"message":null,"error":null}1658gitlab # [ 327.559366] sidekiq-cluster[3410]: {"severity":"INFO","time":"2026-10-03T07:04:47.415Z","message":"Starting cluster with 1 processes"}1659gitlab # [ 330.412880] gitaly[934]: time="2026-10-03T07:04:50.312Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4098VBHWXG2DREH911JVB3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:05:00.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:50.302" grpc.time_ms=10.249 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1660gitlab # [ 332.528102] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:52 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 26471661gitlab # [ 332.530857] setup-gitlab-start[2976]: + mkdir -p /tmp/test-flake1662gitlab # [ 332.532972] setup-gitlab-start[2976]: + cd /tmp/test-flake1663gitlab # [ 332.534309] setup-gitlab-start[2976]: + git init -b master1664gitlab # [ 332.547873] setup-gitlab-start[3523]: Initialized empty Git repository in /tmp/test-flake/.git/1665gitlab # [ 332.549889] setup-gitlab-start[2976]: + git config user.name test1666gitlab # [ 332.551781] setup-gitlab-start[2976]: + git config user.email test@example.com1667gitlab # [ 332.555056] setup-gitlab-start[2976]: + cat1668gitlab # [ 332.557719] setup-gitlab-start[2976]: + git add flake.nix1669gitlab # [ 332.561829] setup-gitlab-start[2976]: + git commit -m 'initial commit'1670gitlab # [ 332.577619] setup-gitlab-start[3528]: [master (root-commit) a0a84e5] initial commit1671gitlab # [ 332.578876] setup-gitlab-start[3528]: 1 file changed, 10 insertions(+)1672gitlab # [ 332.579827] setup-gitlab-start[3528]: create mode 100644 flake.nix1673gitlab # [ 332.581117] setup-gitlab-start[2976]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1674gitlab # [ 332.588536] setup-gitlab-start[2976]: + git push -u origin master1675gitlab # [ 332.650765] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:52 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 381676gitlab # [ 332.803854] gitaly[934]: time="2026-10-03T07:04:52.703Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y282VF7R26GE3VWG9A6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:52.691" grpc.time_ms=11.799 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1677gitlab # [ 332.826468] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:52 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1511678gitlab # [ 333.079084] gitaly[934]: time="2026-10-03T07:04:52.979Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:05:02.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:52.968" grpc.time_ms=10.981 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1679gitlab # [ 333.152087] gitaly[934]: time="2026-10-03T07:04:53.049Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-03T07:05:23.037" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-03T07:04:53.037" grpc.time_ms=11.39 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1680gitlab # [ 333.189693] gitaly[934]: time="2026-10-03T07:04:53.090Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4098Y7CS57X4YGTS4JV543V duration_ms=225ns method=POST pid=934 status=200 url="http://unix/api/v4/internal/allowed"1681gitlab # [ 333.225457] gitaly[934]: time="2026-10-03T07:04:53.125Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4098Y7CS57X4YGTS4JV543V duration_ms=34ns method=POST pid=934 status=200 url="http://unix/api/v4/internal/pre_receive"1682gitlab # [ 333.233652] gitaly[934]: time="2026-10-03T07:04:53.126Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:04:52.863" grpc.time_ms=262.669 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1683gitlab # [ 333.284095] gitaly[934]: time="2026-10-03T07:04:53.184Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:04:53.182" grpc.time_ms=1.312 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1684gitlab # [ 333.339170] gitaly[934]: time="2026-10-03T07:04:53.239Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:04:53.238" grpc.time_ms=0.721 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1685gitlab # [ 333.388098] gitaly[934]: time="2026-10-03T07:04:53.288Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:04:53.287" grpc.time_ms=0.594 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1686gitlab # [ 333.555728] gitaly[934]: time="2026-10-03T07:04:53.455Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:05:03.143" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:53.444" grpc.time_ms=11.66 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1687gitlab # [ 333.593105] gitaly[934]: time="2026-10-03T07:04:53.493Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1431 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=15 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-03T07:05:03.165" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:04:53.466" grpc.time_ms=26.729 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1688gitlab # [ 333.608411] gitaly[934]: time="2026-10-03T07:04:53.507Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4098Y7CS57X4YGTS4JV543V duration_ms=180ns method=POST pid=934 status=200 url="http://unix/api/v4/internal/post_receive"1689gitlab # [ 333.611381] gitaly[934]: time="2026-10-03T07:04:53.509Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:04:53.326" grpc.time_ms=182.373 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1690gitlab # [ 333.624068] gitaly[934]: time="2026-10-03T07:04:53.518Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V diskcache=17514fa1-c8fc-4e64-b640-b7200771dd7c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:52.789" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1691gitlab # [ 333.635489] gitlab-workhorse[820]: time="2026-10-03T07:04:53Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521692gitlab # [ 333.637756] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:53 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 8061693gitlab # [ 333.640106] systemd[1]: Finished setup-gitlab.service.1694gitlab # [ 333.641012] setup-gitlab-start[3532]: To http://gitlab/root/test-flake.git1695gitlab # [ 333.642174] setup-gitlab-start[3532]: * [new branch] master -> master1696gitlab # [ 333.643136] setup-gitlab-start[3532]: branch 'master' set up to track 'origin/master'.1697gitlab # [ 333.644368] gitaly[934]: time="2026-10-03T07:04:53.518Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=238 command.inblock=8 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13820 command.oublock=32 command.real_time_ms=722 command.spawn_token_fork_ms=0 command.system_time_ms=170 command.user_time_ms=68 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:52.789" grpc.time_ms=728.616 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1698gitlab # [ 333.658597] systemd[1]: Starting nixbot CI service...1699gitlab: (finished: waiting for unit setup-gitlab.service, in 334.79 seconds)1700gitlab: waiting for unit nixbot.service1701gitlab # [ 334.829094] nixbot[3609]: {"ts": "2026-10-03T07:04:54.728Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1702gitlab # [ 334.864063] nixbot[3609]: {"ts": "2026-10-03T07:04:54.764Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1703gitlab # [ 334.887898] nixbot[3609]: {"ts": "2026-10-03T07:04:54.788Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1704gitlab # [ 334.892467] nixbot[3609]: {"ts": "2026-10-03T07:04:54.792Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1705gitlab # [ 334.896167] nixbot[3609]: {"ts": "2026-10-03T07:04:54.796Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1706gitlab # [ 334.902175] nixbot[3609]: {"ts": "2026-10-03T07:04:54.802Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1707gitlab # [ 334.906539] nixbot[3609]: {"ts": "2026-10-03T07:04:54.806Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1708gitlab # [ 334.910059] nixbot[3609]: {"ts": "2026-10-03T07:04:54.808Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1709gitlab # [ 334.914023] nixbot[3609]: {"ts": "2026-10-03T07:04:54.814Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1710gitlab # [ 334.917498] nixbot[3609]: {"ts": "2026-10-03T07:04:54.815Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1711gitlab # [ 334.921002] nixbot[3609]: {"ts": "2026-10-03T07:04:54.820Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1712gitlab # [ 334.926646] nixbot[3609]: {"ts": "2026-10-03T07:04:54.826Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1713gitlab # [ 334.930250] nixbot[3609]: {"ts": "2026-10-03T07:04:54.830Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1714gitlab # [ 334.933787] nixbot[3609]: {"ts": "2026-10-03T07:04:54.831Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1715gitlab # [ 334.937532] nixbot[3609]: {"ts": "2026-10-03T07:04:54.835Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1716gitlab # [ 334.941088] nixbot[3609]: {"ts": "2026-10-03T07:04:54.836Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1717gitlab # [ 334.944581] nixbot[3609]: {"ts": "2026-10-03T07:04:54.838Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1718gitlab # [ 334.948107] nixbot[3609]: {"ts": "2026-10-03T07:04:54.839Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1719gitlab # [ 334.951396] nixbot[3609]: {"ts": "2026-10-03T07:04:54.843Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1720gitlab # [ 334.955048] nixbot[3609]: {"ts": "2026-10-03T07:04:54.845Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1721gitlab # [ 334.958702] nixbot[3609]: {"ts": "2026-10-03T07:04:54.846Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1722gitlab # [ 334.962277] nixbot[3609]: {"ts": "2026-10-03T07:04:54.851Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1723gitlab # [ 334.965876] nixbot[3609]: {"ts": "2026-10-03T07:04:54.852Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1724gitlab # [ 334.969331] nixbot[3609]: {"ts": "2026-10-03T07:04:54.854Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1725gitlab # [ 334.972599] nixbot[3609]: {"ts": "2026-10-03T07:04:54.855Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1726gitlab # [ 334.975793] nixbot[3609]: {"ts": "2026-10-03T07:04:54.857Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1727gitlab # [ 334.979352] nixbot[3609]: {"ts": "2026-10-03T07:04:54.859Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1728gitlab # [ 334.982733] nixbot[3609]: {"ts": "2026-10-03T07:04:54.860Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1729gitlab # [ 334.985986] nixbot[3609]: {"ts": "2026-10-03T07:04:54.861Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1730gitlab # [ 334.989084] nixbot[3609]: {"ts": "2026-10-03T07:04:54.872Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1731gitlab # [ 334.992140] nixbot[3609]: {"ts": "2026-10-03T07:04:54.877Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1732gitlab # [ 334.996218] nixbot[3609]: {"ts": "2026-10-03T07:04:54.878Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1733gitlab # [ 335.001184] nixbot[3609]: {"ts": "2026-10-03T07:04:54.879Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1734gitlab # [ 335.004727] nixbot[3609]: {"ts": "2026-10-03T07:04:54.883Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1735gitlab # [ 335.008198] nixbot[3609]: {"ts": "2026-10-03T07:04:54.884Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1736gitlab # [ 335.012216] nixbot[3609]: {"ts": "2026-10-03T07:04:54.889Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1737gitlab # [ 335.015696] nixbot[3609]: {"ts": "2026-10-03T07:04:54.891Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1738gitlab # [ 335.356142] nixbot[3609]: INFO: Started server process [3609]1739gitlab # [ 335.357783] nixbot[3609]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1740gitlab # [ 335.379031] nixbot[3609]: INFO: ::ffff:127.0.0.1:50372 - "GET /health HTTP/1.1" 200 OK1741gitlab # [ 335.385609] systemd[1]: Started nixbot CI service.1742gitlab # [ 335.386844] systemd[1]: Reached target Multi-User System.1743gitlab # [ 335.388523] systemd[1]: Startup finished in 703ms (kernel) + 3.685s (initrd) + 5min 30.999s (userspace) = 5min 35.388s.1744gitlab: (finished: waiting for unit nixbot.service, in 1.06 seconds)1745gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1746gitlab # [ 335.455370] nixbot[3609]: INFO: ::ffff:127.0.0.1:50380 - "GET /health HTTP/1.1" 200 OK1747gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1748subtest: project discovered and webhook registered1749gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1750gitlab # [ 335.583785] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 821751gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds)1752gitlab # [ 335.761982] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:55 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 3841753gitlab # [ 335.766005] nixbot[3609]: {"ts": "2026-10-03T07:04:55.664Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1754gitlab # [ 335.786649] nixbot[3609]: {"ts": "2026-10-03T07:04:55.686Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1755gitlab # [ 335.871064] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 751756gitlab # [ 335.874516] nixbot[3609]: {"ts": "2026-10-03T07:04:55.772Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1757gitlab # [ 335.877506] nixbot[3609]: {"ts": "2026-10-03T07:04:55.774Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1758gitlab # [ 336.009587] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 1311759gitlab # [ 336.012088] nixbot[3609]: {"ts": "2026-10-03T07:04:55.911Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1760gitlab # [ 336.273057] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2511761gitlab # [ 336.277056] nixbot[3609]: {"ts": "2026-10-03T07:04:56.174Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1762gitlab # [ 336.415807] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 1271763gitlab # [ 336.419653] nixbot[3609]: {"ts": "2026-10-03T07:04:56.311Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1764gitlab # [ 336.492526] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 761765gitlab # [ 336.495990] nixbot[3609]: {"ts": "2026-10-03T07:04:56.393Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1766gitlab # [ 336.566387] gitaly[934]: time="2026-10-03T07:04:56.466Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40991TF9RM559HFXMCVDN84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-03T07:05:26.049" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:04:56.449" grpc.time_ms=15.807 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1767gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1768gitlab # [ 336.583219] gitaly[934]: time="2026-10-03T07:04:56.481Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40991TF9RM559HFXMCVDN84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:05:26.080" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:04:56.480" grpc.time_ms=0.803 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1769gitlab # [ 336.601118] gitaly[934]: time="2026-10-03T07:04:56.501Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40991TF9RM559HFXMCVDN84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-03T07:05:06.084" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:04:56.484" grpc.time_ms=16.571 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1770gitlab # [ 336.637435] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1371771gitlab # [ 336.640195] nixbot[3609]: {"ts": "2026-10-03T07:04:56.538Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1772gitlab # [ 336.710564] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 641773gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.13 seconds)1774(finished: subtest: project discovered and webhook registered, in 1.25 seconds)1775subtest: push delivers the webhook1776gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1777gitlab # [ 336.764342] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1201778gitlab # [ 336.767754] nixbot[3609]: {"ts": "2026-10-03T07:04:56.666Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1779gitlab # [ 336.771099] nixbot[3609]: {"ts": "2026-10-03T07:04:56.670Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1780gitlab # [ 336.918281] gitaly[934]: time="2026-10-03T07:04:56.818Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4099259S36XD3K2BE9844K2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:56.806" grpc.time_ms=11.542 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1781gitlab # [ 336.941661] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 731782gitlab # [ 337.033229] gitaly[934]: time="2026-10-03T07:04:56.933Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M409927M01EZZRKSSS4Y815X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:56.920" pid=934 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1783gitlab # [ 337.050488] gitaly[934]: time="2026-10-03T07:04:56.933Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M409927M01EZZRKSSS4Y815X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:56.920" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 337.071889] gitlab-workhorse[820]: time="2026-10-03T07:04:56Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131785gitlab # [ 337.075160] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:56 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1131786gitlab # [ 337.078082] gitaly[934]: time="2026-10-03T07:04:56.934Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M409927M01EZZRKSSS4Y815X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:56.920" grpc.time_ms=13.188 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1787gitlab # [ 337.275535] gitaly[934]: time="2026-10-03T07:04:57.175Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M40992BA9Q49EHTYF8CZQ6GQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.154" pid=934 remote_ip="2001:db8:1::1" request_sha=6daf7915a65550ccbdafad2fdf700a0f6160e1e6 response_bytes=223 span.kind=server system=grpc1788gitlab # [ 337.293964] gitaly[934]: time="2026-10-03T07:04:57.176Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M40992BA9Q49EHTYF8CZQ6GQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.154" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1789gitlab # [ 337.316655] gitlab-workhorse[820]: time="2026-10-03T07:04:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231790gitlab # [ 337.319717] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 2371791gitlab # [ 337.323104] gitaly[934]: time="2026-10-03T07:04:57.176Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1136 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M40992BA9Q49EHTYF8CZQ6GQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.154" grpc.time_ms=21.89 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1792gitlab # [ 337.384111] gitaly[934]: time="2026-10-03T07:04:57.284Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M40992M59W6222S4D3ZTYBCJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.273" grpc.time_ms=10.41 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1793gitlab # [ 337.405939] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:57 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 631794gitlab # [ 337.451803] gitaly[934]: time="2026-10-03T07:04:57.352Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M40992P6DZJJ2WG5V8SWF79Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.340" pid=934 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1795gitlab # [ 337.468739] gitaly[934]: time="2026-10-03T07:04:57.352Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40992P6DZJJ2WG5V8SWF79Y grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.340" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1796gitlab # [ 337.491769] gitaly[934]: time="2026-10-03T07:04:57.352Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40992P6DZJJ2WG5V8SWF79Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.340" grpc.time_ms=11.572 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1797gitlab # [ 337.511439] gitlab-workhorse[820]: time="2026-10-03T07:04:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671798gitlab # [ 337.513732] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 821799gitlab # [ 337.596657] gitaly[934]: time="2026-10-03T07:04:57.496Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M40992TJVB6Z9K7DWV9KPJTQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.485" grpc.time_ms=10.206 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1800gitlab # [ 337.618772] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:57 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 701801gitlab # [ 337.677637] gitaly[934]: time="2026-10-03T07:04:57.577Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M40992WVGYW24FQ3NGEQASKF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.556" pid=934 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1802gitlab # [ 337.689688] gitaly[934]: time="2026-10-03T07:04:57.589Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40992WVGYW24FQ3NGEQASKF grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.556" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1803gitlab # [ 337.705523] gitaly[934]: time="2026-10-03T07:04:57.589Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40992WVGYW24FQ3NGEQASKF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:04:57.556" grpc.time_ms=33.129 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1804gitlab # [ 337.721806] gitlab-workhorse[820]: time="2026-10-03T07:04:57Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091805gitlab # [ 337.723785] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:57 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 911806gitlab # [ 337.810181] gitaly[934]: time="2026-10-03T07:04:57.710Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:05:27.709" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:04:57.709" grpc.time_ms=0.816 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1807gitlab # [ 337.876808] gitaly[934]: time="2026-10-03T07:04:57.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-03T07:05:27.766" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:04:57.766" grpc.time_ms=10.444 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1808gitlab # [ 337.904344] gitaly[934]: time="2026-10-03T07:04:57.804Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:05:07.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:04:57.795" grpc.time_ms=9.397 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1809gitlab # [ 337.927701] gitaly[934]: time="2026-10-03T07:04:57.828Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:05:07.026" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:04:57.816" grpc.time_ms=11.209 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1810gitlab # [ 338.041827] gitaly[934]: time="2026-10-03T07:04:57.942Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-03T07:05:07.151" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:57.941" grpc.time_ms=0.439 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1811gitlab # [ 338.063831] gitaly[934]: time="2026-10-03T07:04:57.964Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-03T07:05:07.162" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:04:57.953" grpc.time_ms=10.629 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1812gitlab # [ 338.116046] gitaly[934]: time="2026-10-03T07:04:58.016Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:04:58.015" grpc.time_ms=0.473 pid=934 span.kind=server system=grpc1813gitlab # [ 338.165932] gitaly[934]: time="2026-10-03T07:04:58.066Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:04:58.065" grpc.time_ms=0.492 pid=934 span.kind=server system=grpc1814gitlab # [ 338.182176] gitaly[934]: time="2026-10-03T07:04:58.079Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY diskcache=bab7997d-d044-4248-8a73-2303beb9f45e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-03T07:05:07.006" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:57.967" pid=934 ref=refs/keep-around/a0a84e5503e4e6dbd339b9250ae18211495d317e remote_ip="2001:db8:1::1" revision=a0a84e5503e4e6dbd339b9250ae18211495d317e span.kind=server system=grpc username=root1815gitlab # [ 338.200299] gitaly[934]: time="2026-10-03T07:04:58.080Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=88 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5321 command.oublock=8 command.real_time_ms=107 command.spawn_token_fork_ms=1 command.system_time_ms=40 command.user_time_ms=48 component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-03T07:05:07.006" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:04:57.967" grpc.time_ms=112.682 pid=934 ref=refs/keep-around/a0a84e5503e4e6dbd339b9250ae18211495d317e remote_ip="2001:db8:1::1" revision=a0a84e5503e4e6dbd339b9250ae18211495d317e span.kind=server system=grpc username=root1816gitlab # [ 338.220205] gitaly[934]: time="2026-10-03T07:04:58.084Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:05:28.083" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:04:58.084" grpc.time_ms=0.419 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1817gitlab # [ 338.391894] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:04:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6501818gitlab # [ 338.396997] nixbot[3609]: {"ts": "2026-10-03T07:04:58.293Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e \"HTTP/1.1 201 Created\""}1819gitlab # [ 338.404913] nixbot[3609]: {"ts": "2026-10-03T07:04:58.305Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-a0a84e5503e4-0"}1820gitlab # [ 338.602400] nixbot[3609]: {"ts": "2026-10-03T07:04:58.502Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-a0a84e5503e4-0", "argv0": "bwrap"}1821gitlab # [ 338.758983] systemd[1]: Started Nix Daemon.1822gitlab # [ 338.854338] nix-daemon[3765]: accepted connection from pid 3760, user nixbot1823gitlab # [ 342.912495] nix-daemon[3765]: accepted connection from pid 3804, user nixbot1824gitlab # [ 342.929263] nix-daemon[3765]: accepted connection from pid 3760, user nixbot1825gitlab # [ 347.290096] nix-daemon[3765]: accepted connection from pid 3760, user nixbot1826gitlab # [ 347.429201] gitaly[934]: time="2026-10-03T07:05:07.328Z" level=info msg="finished unary call with code OK" catfile.duration_ms=4 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=4 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4099CDG0NRMGGKATT16FH4J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:05:37.321" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:05:07.322" grpc.time_ms=6.593 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827gitlab # [ 347.477715] gitaly[934]: time="2026-10-03T07:05:07.377Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4099CDG0NRMGGKATT16FH4J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:05:17.064" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:05:07.364" grpc.time_ms=13.019 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1828gitlab # [ 347.544861] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:05:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1961829gitlab # [ 347.549621] nixbot[3609]: {"ts": "2026-10-03T07:05:07.446Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e \"HTTP/1.1 201 Created\""}1830gitlab # [ 347.617064] gitaly[934]: time="2026-10-03T07:05:07.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4099CKVM9QPCTXFCZQ4T9EK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:05:37.515" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:05:07.516" grpc.time_ms=0.818 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1831gitlab # [ 347.683862] gitaly[934]: time="2026-10-03T07:05:07.584Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4099CKVM9QPCTXFCZQ4T9EK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:05:17.270" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:05:07.571" grpc.time_ms=12.83 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1832gitlab # [ 347.747860] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:05:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1961833gitlab # [ 347.752422] nixbot[3609]: {"ts": "2026-10-03T07:05:07.649Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e \"HTTP/1.1 201 Created\""}1834gitlab # [ 347.826755] gitaly[934]: time="2026-10-03T07:05:07.726Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4099CTEHPB16T7C8N7ECYYY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:05:37.725" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:05:07.726" grpc.time_ms=0.698 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1835gitlab # [ 347.867998] gitaly[934]: time="2026-10-03T07:05:07.768Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4099CTEHPB16T7C8N7ECYYY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:05:17.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:05:07.757" grpc.time_ms=11.161 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1836gitlab # [ 347.938964] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:05:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1761837gitlab # [ 347.943511] nixbot[3609]: {"ts": "2026-10-03T07:05:07.840Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e \"HTTP/1.1 201 Created\""}1838gitlab # [ 348.363600] gitaly[934]: time="2026-10-03T07:05:08.263Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4099DATQARPMMRNR2861Z7F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.250" grpc.time_ms=12.675 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1839gitlab # [ 348.386175] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:05:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 771840gitlab # [ 348.439837] gitaly[934]: time="2026-10-03T07:05:08.339Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4099DDD58X57ZX6V8ECJZ2M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.327" pid=934 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1841gitlab # [ 348.453359] gitaly[934]: time="2026-10-03T07:05:08.339Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M4099DDD58X57ZX6V8ECJZ2M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.327" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1842gitlab # [ 348.468771] gitlab-workhorse[820]: time="2026-10-03T07:05:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671843gitlab # [ 348.470830] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:05:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 701844gitlab # [ 348.475257] gitaly[934]: time="2026-10-03T07:05:08.340Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M4099DDD58X57ZX6V8ECJZ2M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.327" grpc.time_ms=12.37 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1845gitlab # [ 348.584991] gitaly[934]: time="2026-10-03T07:05:08.485Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4099DH24R4XXGYGFH3HNR45 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.474" grpc.time_ms=10.213 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1846gitlab # [ 348.606947] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:05:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 981847gitlab # [ 348.678786] gitaly[934]: time="2026-10-03T07:05:08.578Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4099DM7XZW9ARZ7EYSHJ1NB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.566" pid=934 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1848gitlab # [ 348.695811] gitaly[934]: time="2026-10-03T07:05:08.578Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4099DM7XZW9ARZ7EYSHJ1NB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.566" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1849gitlab # [ 348.716710] gitlab-workhorse[820]: time="2026-10-03T07:05:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671850gitlab # [ 348.719366] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:05:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 921851gitlab # [ 348.722099] gitaly[934]: time="2026-10-03T07:05:08.578Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4099DM7XZW9ARZ7EYSHJ1NB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:05:08.566" grpc.time_ms=12.179 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1852gitlab # [ 373.623036] sidekiq-cluster[3482]: 2026-10-03T07:05:33.522Z pid=3482 tid=ylu INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3482, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430817280}1853gitlab # [ 373.937052] sidekiq-cluster[3482]: W, [2026-10-03T07:05:33.837350 #3482] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1854gitlab # [ 374.004441] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:05:33.904Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1855gitlab # [ 377.689639] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:05:37.589Z","message":"GitLab reliable fetch activated!"}1856gitlab # [ 377.689923] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:05:37.589Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1857gitlab # [ 415.578245] postgres[4408]: [4408] ERROR: relation "table_without_models" does not exist1858gitlab # [ 415.578630] postgres[4408]: [4408] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1859gitlab # [ 420.448207] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:20.347Z","message":"Booted Rails 7.2.3.1 application in production environment"}1860gitlab # [ 420.452543] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:20.352Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1861gitlab # [ 420.454613] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:20.353Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1862gitlab # [ 420.456425] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:20.353Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1863gitlab # [ 420.458470] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:20.355Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1864gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 84.16 seconds)1865gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1866gitlab # [ 420.998644] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:20 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 711867gitlab # [ 421.180376] gitaly[934]: time="2026-10-03T07:06:21.079Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BMB3SWN059N83QMRPFBR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:21.065" grpc.time_ms=13.925 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1868gitlab # Enumerating objects: 5, done.1869gitlab # [ 421.203357] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:21 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 1801870gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1871gitlab # Delta compression using up to 4 threads1872gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1873gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 273.00 KiB/s, done.1874gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1875gitlab # [ 421.494019] gitaly[934]: time="2026-10-03T07:06:21.394Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-03T07:06:51.081" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-03T07:06:21.381" grpc.time_ms=12.238 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 421.519470] gitaly[934]: time="2026-10-03T07:06:21.419Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ duration_ms=136ns method=POST pid=934 status=200 url="http://unix/api/v4/internal/allowed"1877gitlab # [ 421.557776] gitaly[934]: time="2026-10-03T07:06:21.458Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ duration_ms=37ns method=POST pid=934 status=200 url="http://unix/api/v4/internal/pre_receive"1878gitlab # [ 421.565503] gitaly[934]: time="2026-10-03T07:06:21.458Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:21.282" grpc.time_ms=176.338 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1879gitlab # [ 421.618849] gitaly[934]: time="2026-10-03T07:06:21.519Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:21.518" grpc.time_ms=0.9 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1880gitlab # [ 421.659983] gitaly[934]: time="2026-10-03T07:06:21.560Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:21.559" grpc.time_ms=0.875 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1881gitlab # [ 421.725582] gitaly[934]: time="2026-10-03T07:06:21.625Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:21.624" grpc.time_ms=0.848 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1882gitlab # [ 421.924423] gitaly[934]: time="2026-10-03T07:06:21.824Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ duration_ms=128ns method=POST pid=934 status=200 url="http://unix/api/v4/internal/post_receive"1883gitlab # [ 421.932732] gitaly[934]: time="2026-10-03T07:06:21.825Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:21.695" grpc.time_ms=130.205 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1884gitlab # [ 421.954995] gitaly[934]: time="2026-10-03T07:06:21.855Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ diskcache=04edcdc9-3fe2-4abf-b827-4c73caa9d31c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:21.213" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1885gitlab # To http://gitlab/root/test-flake.git1886gitlab # [ 421.971732] gitaly[934]: time="2026-10-03T07:06:21.855Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=277 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13958 command.oublock=32 command.real_time_ms=635 command.spawn_token_fork_ms=0 command.system_time_ms=143 command.user_time_ms=134 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:21.213" grpc.time_ms=642.153 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1887gitlab # a0a84e5..e874471 master -> master1888gitlab # [ 422.000364] gitlab-workhorse[820]: time="2026-10-03T07:06:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521889gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.14 seconds)1890gitlab # [ 422.002914] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:21 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7381891gitlab: must succeed: git -C /tmp/test-flake rev-parse master1892gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1893gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'1894gitlab # [ 422.148558] nixbot[3609]: INFO: ::ffff:127.0.0.1:44158 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK1895gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.13 seconds)1896gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'1897gitlab # [ 423.186351] nixbot[3609]: INFO: ::ffff:127.0.0.1:44172 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK1898gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.04 seconds)1899gitlab # [ 423.274901] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.174Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"31fbf07571cb8abb5ec96742","created_at":"2026-10-03T07:04:43.240Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:43.288Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-31fbf07571cb8abb5ec96742: start","job_status":"start","queue_duration_s":99.885327,"scheduling_latency_s":99.885388}1900gitlab # [ 423.512633] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.412Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"31fbf07571cb8abb5ec96742","created_at":"2026-10-03T07:04:43.240Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:43.288Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-31fbf07571cb8abb5ec96742: done: 0.238275 sec","job_status":"done","queue_duration_s":99.885327,"scheduling_latency_s":99.885388,"redis_calls":23,"redis_duration_s":0.005992,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003076,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001621,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.120434,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12243172,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.238275,"completed_at":"2026-10-03T07:06:23.412Z","load_balancing_strategy":"primary","db_duration_s":0.063863,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1901gitlab # [ 423.580505] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.414Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"fc7cc6074f76f2485ab4f32d","created_at":"2026-10-03T07:04:43.300Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:43.306Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-fc7cc6074f76f2485ab4f32d: start","job_status":"start","queue_duration_s":100.107548,"scheduling_latency_s":100.107575}1902gitlab # [ 423.650339] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.550Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"fc7cc6074f76f2485ab4f32d","created_at":"2026-10-03T07:04:43.300Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:43.306Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-fc7cc6074f76f2485ab4f32d: done: 0.135608 sec","job_status":"done","queue_duration_s":100.107548,"scheduling_latency_s":100.107575,"redis_calls":14,"redis_duration_s":0.003845,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000793,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001869,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10234394,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.135608,"completed_at":"2026-10-03T07:06:23.549Z","load_balancing_strategy":"primary","db_duration_s":0.022594,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1903gitlab # [ 423.716504] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.552Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"27777a1364240276082b12c6","created_at":"2026-10-03T07:04:43.959Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:43.968Z","job_size_bytes":5,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-27777a1364240276082b12c6: start","job_status":"start","queue_duration_s":99.583794,"scheduling_latency_s":99.583819}1904gitlab # [ 423.835749] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.735Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"27777a1364240276082b12c6","created_at":"2026-10-03T07:04:43.959Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:43.968Z","job_size_bytes":5,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-27777a1364240276082b12c6: done: 0.183472 sec","job_status":"done","queue_duration_s":99.583794,"scheduling_latency_s":99.583819,"redis_calls":12,"redis_duration_s":0.0030239999999999998,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000694,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001577,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000753,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.149775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044656,"duration_s":0.183472,"completed_at":"2026-10-03T07:06:23.735Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027311,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1905gitlab # [ 423.907461] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.737Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"51759bd11fade430d30bf568","created_at":"2026-10-03T07:04:43.999Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:44.005Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-51759bd11fade430d30bf568: start","job_status":"start","queue_duration_s":99.73231,"scheduling_latency_s":99.732329}1906gitlab # [ 424.049325] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.949Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"51759bd11fade430d30bf568","created_at":"2026-10-03T07:04:43.999Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:44.005Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-51759bd11fade430d30bf568: done: 0.211839 sec","job_status":"done","queue_duration_s":99.73231,"scheduling_latency_s":99.732329,"redis_calls":11,"redis_duration_s":0.002513,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00066,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001363,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00049,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.175,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.175,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.148579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038716,"duration_s":0.211839,"completed_at":"2026-10-03T07:06:23.949Z","load_balancing_strategy":"primary","db_duration_s":0.066551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1907gitlab # [ 424.116980] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.950Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"7bf06e0e7efefd30a0ec074b","created_at":"2026-10-03T07:04:44.008Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:44.014Z","job_size_bytes":89,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-7bf06e0e7efefd30a0ec074b: start","job_status":"start","queue_duration_s":99.936537,"scheduling_latency_s":99.936556}1908gitlab # [ 424.137392] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.989Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"7bf06e0e7efefd30a0ec074b","created_at":"2026-10-03T07:04:44.008Z","trace_propagation_headers":{"sentry-trace":"8276c01d448f404e962b60546262a8f4-57d1e0bbe21c4e68","baggage":"sentry-trace_id=8276c01d448f404e962b60546262a8f4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098KXNS0N7VPMTNG97FBGY","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:44.014Z","job_size_bytes":89,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-7bf06e0e7efefd30a0ec074b: done: 0.039056 sec","job_status":"done","queue_duration_s":99.936537,"scheduling_latency_s":99.936556,"redis_calls":11,"redis_duration_s":0.002266,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000592,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00124,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003243,"duration_s":0.039056,"completed_at":"2026-10-03T07:06:23.989Z","load_balancing_strategy":"primary","db_duration_s":0.005681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1909gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'1910gitlab # [ 424.184432] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:23.991Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 a0a84e5503e4e6dbd339b9250ae18211495d317e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ed1d42a4a50cc65f1a65adec","created_at":"2026-10-03T07:04:53.372Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-35601cca642640d8","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:53.408Z","job_size_bytes":185,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ed1d42a4a50cc65f1a65adec: start","job_status":"start","queue_duration_s":90.582967,"scheduling_latency_s":90.582985}1911gitlab # [ 424.201713] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:24.007Z","message":"changes: \"0000000000000000000000000000000000000000 a0a84e5503e4e6dbd339b9250ae18211495d317e refs/heads/master\\n\""}1912gitlab # [ 424.204614] gitaly[934]: time="2026-10-03T07:06:24.050Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:34.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:24.038" grpc.time_ms=12.055 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 424.218864] gitaly[934]: time="2026-10-03T07:06:24.066Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:06:34.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:24.055" grpc.time_ms=10.327 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.04 seconds)1915gitlab # [ 424.233300] gitaly[934]: time="2026-10-03T07:06:24.127Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=6 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-03T07:06:34.084" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:24.115" grpc.time_ms=12.455 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 424.247885] nixbot[3609]: INFO: ::ffff:127.0.0.1:44186 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK1917gitlab # [ 424.275171] gitaly[934]: time="2026-10-03T07:06:24.175Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:24.175" grpc.time_ms=0.407 pid=934 span.kind=server system=grpc1918gitlab # [ 424.312865] gitaly[934]: time="2026-10-03T07:06:24.213Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:24.212" grpc.time_ms=0.487 pid=934 span.kind=server system=grpc1919gitlab # [ 424.327822] gitaly[934]: time="2026-10-03T07:06:24.223Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V diskcache=329393ef-bb0c-48b2-84b5-ffef405facd9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-03T07:06:34.107" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:24.138" pid=934 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1920gitlab # [ 424.342063] gitaly[934]: time="2026-10-03T07:06:24.223Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=86 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5149 command.oublock=8 command.real_time_ms=81 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=29 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-03T07:06:34.107" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:24.138" grpc.time_ms=85.38 pid=934 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1921gitlab # [ 424.358706] gitaly[934]: time="2026-10-03T07:06:24.239Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-03T07:06:34.209" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:24.239" grpc.time_ms=0.184 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922gitlab # [ 424.370500] gitaly[934]: time="2026-10-03T07:06:24.259Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=860 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-03T07:06:34.213" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:24.243" grpc.time_ms=15.553 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab # [ 424.472701] gitaly[934]: time="2026-10-03T07:06:24.373Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-03T07:06:54.065" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:24.365" grpc.time_ms=7.159 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 424.509701] gitaly[934]: time="2026-10-03T07:06:24.410Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-03T07:06:34.009" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:24.409" grpc.time_ms=0.301 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab # [ 424.526799] gitaly[934]: time="2026-10-03T07:06:24.419Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=317 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-03T07:06:34.012" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-03T07:06:24.412" grpc.time_ms=6.755 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1926gitlab # [ 424.547913] gitaly[934]: time="2026-10-03T07:06:24.446Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:34.035" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:24.435" grpc.time_ms=10.881 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927gitlab # [ 424.565597] gitaly[934]: time="2026-10-03T07:06:24.463Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:34.052" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:24.452" grpc.time_ms=10.415 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab # [ 424.593607] gitaly[934]: time="2026-10-03T07:06:24.493Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-03T07:06:54.177" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:24.477" grpc.time_ms=16.58 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 425.126910] gitaly[934]: time="2026-10-03T07:06:25.027Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-03T07:06:35.004" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:25.025" grpc.time_ms=2.01 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'1931gitlab # [ 425.172760] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.071Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 a0a84e5503e4e6dbd339b9250ae18211495d317e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"ed1d42a4a50cc65f1a65adec","created_at":"2026-10-03T07:04:53.372Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-35601cca642640d8","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:04:53.408Z","job_size_bytes":185,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-ed1d42a4a50cc65f1a65adec: done: 1.079917 sec","job_status":"done","queue_duration_s":90.582967,"scheduling_latency_s":90.582985,"gitaly_calls":13,"gitaly_duration_s":0.229118,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.041315000000000004,"redis_read_bytes":2103,"redis_write_bytes":21748,"redis_cache_calls":15,"redis_cache_duration_s":0.00385,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005823,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001505,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7836,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004296,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.015542,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010299,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.289,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.322,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.753558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063968,"exclusive_lock_hold_duration_s":0.032746500000000005,"duration_s":1.079917,"completed_at":"2026-10-03T07:06:25.071Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.103058,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1932gitlab # [ 425.247438] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.073Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"3f5331ba6525a1ce604bf483","created_at":"2026-10-03T07:04:58.093Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-32d51393972d4b5a","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:58.102Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-3f5331ba6525a1ce604bf483: start","job_status":"start","queue_duration_s":86.971058,"scheduling_latency_s":86.971088}1933gitlab # [ 425.276559] nixbot[3609]: INFO: ::ffff:127.0.0.1:44192 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK1934gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.04 seconds)1935gitlab # [ 425.291155] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.190Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"3f5331ba6525a1ce604bf483","created_at":"2026-10-03T07:04:58.093Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-32d51393972d4b5a","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:58.102Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-3f5331ba6525a1ce604bf483: done: 0.117043 sec","job_status":"done","queue_duration_s":86.971058,"scheduling_latency_s":86.971088,"redis_calls":16,"redis_duration_s":0.005794,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003567,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001574,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000653,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00048468,"duration_s":0.117043,"completed_at":"2026-10-03T07:06:25.190Z","load_balancing_strategy":"primary","db_duration_s":0.025933,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1936gitlab # [ 425.356346] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.193Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9d833e99bed1c0756c9fe092","created_at":"2026-10-03T07:04:58.104Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-32d51393972d4b5a","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:58.111Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9d833e99bed1c0756c9fe092: start","job_status":"start","queue_duration_s":87.081405,"scheduling_latency_s":87.081427}1937gitlab # [ 425.408342] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.308Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"9d833e99bed1c0756c9fe092","created_at":"2026-10-03T07:04:58.104Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-32d51393972d4b5a","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:58.111Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-9d833e99bed1c0756c9fe092: done: 0.115075 sec","job_status":"done","queue_duration_s":87.081405,"scheduling_latency_s":87.081427,"redis_calls":21,"redis_duration_s":0.004567,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002085,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031394,"duration_s":0.115075,"completed_at":"2026-10-03T07:06:25.307Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1938gitlab # [ 425.471432] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.309Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"90a69063934bd607042399d9","created_at":"2026-10-03T07:04:58.264Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-32d51393972d4b5a","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:58.271Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-90a69063934bd607042399d9: start","job_status":"start","queue_duration_s":87.037912,"scheduling_latency_s":87.037932}1939gitlab # [ 425.717303] gitaly[934]: time="2026-10-03T07:06:25.617Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:25.615" grpc.time_ms=1.739 pid=934 span.kind=server system=grpc1940gitlab # [ 425.751955] gitaly[934]: time="2026-10-03T07:06:25.652Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:25.651" grpc.time_ms=0.59 pid=934 span.kind=server system=grpc1941gitlab # [ 425.785228] gitaly[934]: time="2026-10-03T07:06:25.685Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:25.684" grpc.time_ms=0.558 pid=934 span.kind=server system=grpc1942gitlab # [ 425.797662] gitaly[934]: time="2026-10-03T07:06:25.693Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY diskcache=5ed4ac3e-eaa9-4dde-92a1-62e379db55e8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:06:55.279" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:25.580" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1943gitlab # [ 425.808856] gitaly[934]: time="2026-10-03T07:06:25.693Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=125 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7531 command.oublock=0 command.real_time_ms=108 command.spawn_token_fork_ms=0 command.system_time_ms=64 command.user_time_ms=61 component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:06:55.279" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:25.580" grpc.time_ms=113.646 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1944gitlab # [ 425.928060] gitaly[934]: time="2026-10-03T07:06:25.828Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-03T07:06:55.526" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:25.827" grpc.time_ms=1.199 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1945gitlab # [ 425.978483] gitaly[934]: time="2026-10-03T07:06:25.878Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M40993189A8N6MJSR9D8F3AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:55.577" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:25.877" grpc.time_ms=0.933 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1946gitlab # [ 426.002878] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.902Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"90a69063934bd607042399d9","created_at":"2026-10-03T07:04:58.264Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-32d51393972d4b5a","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-03T07:04:58.271Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-90a69063934bd607042399d9: done: 0.592928 sec","job_status":"done","queue_duration_s":87.037912,"scheduling_latency_s":87.037932,"gitaly_calls":3,"gitaly_duration_s":0.124002,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025005,"redis_read_bytes":455,"redis_write_bytes":28000,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00127,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001403,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008624,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.003826,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14494,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.005348,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000636,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000543,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.183,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.024,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.153,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.392088,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.40311838,"duration_s":0.592928,"completed_at":"2026-10-03T07:06:25.902Z","load_balancing_strategy":"primary","db_duration_s":0.194528,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1947gitlab # [ 426.072061] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.904Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","a0a84e5503e4e6dbd339b9250ae18211495d317e e874471c6bba3c76d684eaabbff376e5652e08d0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5a1a5a3b780e1d1d81d55f0a","created_at":"2026-10-03T07:06:21.771Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-54e4a44168414cf3","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:21.777Z","job_size_bytes":185,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5a1a5a3b780e1d1d81d55f0a: start","job_status":"start","queue_duration_s":4.126605,"scheduling_latency_s":4.126629}1948gitlab # [ 426.089600] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.919Z","message":"changes: \"a0a84e5503e4e6dbd339b9250ae18211495d317e e874471c6bba3c76d684eaabbff376e5652e08d0 refs/heads/master\\n\""}1949gitlab # [ 426.092101] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:25.971Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"674a327a23a7c1f4ea2994d5","created_at":"2026-10-03T07:06:25.969Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-c45d721aa5c549ed","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"1d68890a46ee8ee483755b07","job_size_bytes":46,"pid":3482,"job_status":"deduplicated","message":"ProjectCacheWorker JID-674a327a23a7c1f4ea2994d5: deduplicated: until executing","deduplication.type":"until executing"}1950gitlab # [ 426.108792] gitaly[934]: time="2026-10-03T07:06:25.950Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:35.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:25.939" grpc.time_ms=11.556 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1951gitlab # [ 426.122496] gitaly[934]: time="2026-10-03T07:06:25.966Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-03T07:06:35.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:25.955" grpc.time_ms=11.098 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1952gitlab # [ 426.136515] gitaly[934]: time="2026-10-03T07:06:25.996Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-03T07:06:35.055" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:25.985" grpc.time_ms=10.556 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1953gitlab # [ 426.150564] gitaly[934]: time="2026-10-03T07:06:26.014Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-03T07:06:56.001" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.001" grpc.time_ms=12.228 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1954gitlab # [ 426.166428] gitaly[934]: time="2026-10-03T07:06:26.032Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-03T07:06:36.005" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-03T07:06:26.025" grpc.time_ms=6.762 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1955gitlab # [ 426.180208] gitaly[934]: time="2026-10-03T07:06:26.053Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:36.023" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:26.043" grpc.time_ms=10.167 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1956gitlab # [ 426.247992] gitaly[934]: time="2026-10-03T07:06:26.148Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-03T07:06:36.126" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.146" grpc.time_ms=1.406 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1957gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'1958gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.04 seconds)1959gitlab # [ 426.269316] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.168Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","a0a84e5503e4e6dbd339b9250ae18211495d317e e874471c6bba3c76d684eaabbff376e5652e08d0 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"5a1a5a3b780e1d1d81d55f0a","created_at":"2026-10-03T07:06:21.771Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-54e4a44168414cf3","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:21.777Z","job_size_bytes":185,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-5a1a5a3b780e1d1d81d55f0a: done: 0.263592 sec","job_status":"done","queue_duration_s":4.126605,"scheduling_latency_s":4.126629,"gitaly_calls":7,"gitaly_duration_s":0.082598,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.021821,"redis_read_bytes":501,"redis_write_bytes":11939,"redis_cache_calls":9,"redis_cache_duration_s":0.002295,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000899,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5716,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003591,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.006761,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.008275,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.027,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.027,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.156063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00097616,"exclusive_lock_hold_duration_s":0.02176722,"duration_s":0.263592,"completed_at":"2026-10-03T07:06:26.167Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024222,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1960gitlab # [ 426.345197] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.169Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1d68890a46ee8ee483755b07","created_at":"2026-10-03T07:06:24.068Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.086Z","job_size_bytes":46,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1d68890a46ee8ee483755b07: start","job_status":"start","queue_duration_s":2.082642,"scheduling_latency_s":2.082662}1961gitlab # [ 426.364714] gitaly[934]: time="2026-10-03T07:06:26.235Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-03T13:06:26.230" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:26.231" grpc.time_ms=4.317 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1962gitlab # [ 426.377154] nixbot[3609]: INFO: ::ffff:127.0.0.1:44198 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK1963gitlab # [ 426.379373] gitaly[934]: time="2026-10-03T07:06:26.256Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-03T07:06:56.244" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.245" grpc.time_ms=10.767 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1964gitlab # [ 426.393928] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.286Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"1d68890a46ee8ee483755b07","created_at":"2026-10-03T07:06:24.068Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.086Z","job_size_bytes":46,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1d68890a46ee8ee483755b07: done: 0.116948 sec","job_status":"done","queue_duration_s":2.082642,"scheduling_latency_s":2.082662,"gitaly_calls":2,"gitaly_duration_s":0.020369,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007744,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001322,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000575,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001517,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003668,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000662,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.075857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032546,"duration_s":0.116948,"completed_at":"2026-10-03T07:06:26.286Z","load_balancing_strategy":"primary","db_duration_s":0.015187,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1965gitlab # [ 426.444374] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.288Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8f90c6ede3a627f0da910a80","created_at":"2026-10-03T07:06:24.103Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.110Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8f90c6ede3a627f0da910a80: start","job_status":"start","queue_duration_s":2.1771,"scheduling_latency_s":2.177118}1966gitlab # [ 426.461499] gitaly[934]: time="2026-10-03T07:06:26.335Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V diskcache=d6777af0-6198-47eb-8f7a-bb9e3bafa8ac grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-03T13:06:26.312" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.313" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 426.471797] gitaly[934]: time="2026-10-03T07:06:26.336Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1171 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-03T13:06:26.312" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.313" grpc.time_ms=22.589 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 426.488708] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.388Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8f90c6ede3a627f0da910a80","created_at":"2026-10-03T07:06:24.103Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.110Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8f90c6ede3a627f0da910a80: done: 0.100439 sec","job_status":"done","queue_duration_s":2.1771,"scheduling_latency_s":2.177118,"gitaly_calls":1,"gitaly_duration_s":0.024861,"redis_calls":15,"redis_duration_s":0.003379,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000427,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001297,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000529,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001126,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.026,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07132476,"duration_s":0.100439,"completed_at":"2026-10-03T07:06:26.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1969gitlab # [ 426.537730] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.390Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0a3b9639e4fcc454152454c1","created_at":"2026-10-03T07:06:24.234Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.236Z","job_size_bytes":23,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0a3b9639e4fcc454152454c1: start","job_status":"start","queue_duration_s":2.15385,"scheduling_latency_s":2.153875}1970gitlab # [ 426.553766] gitaly[934]: time="2026-10-03T07:06:26.435Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-03T07:06:56.424" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.425" grpc.time_ms=10.15 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1971gitlab # [ 426.567522] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.447Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0a3b9639e4fcc454152454c1","created_at":"2026-10-03T07:06:24.234Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.236Z","job_size_bytes":23,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0a3b9639e4fcc454152454c1: done: 0.056487 sec","job_status":"done","queue_duration_s":2.15385,"scheduling_latency_s":2.153875,"gitaly_calls":1,"gitaly_duration_s":0.012322,"redis_calls":16,"redis_duration_s":0.004478,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000339,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002203,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000769,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056487,"completed_at":"2026-10-03T07:06:26.447Z","load_balancing_strategy":"primary","db_duration_s":0.003219,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1972gitlab # [ 426.614190] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:26.449Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"60a07b928b7d17f0edbad23c","created_at":"2026-10-03T07:06:24.840Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ee28d6f97cac7facd39a4e2899b3a54ee1a15b32fb6fd96f32a97a8fbfbf5627","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.846Z","job_size_bytes":467,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-60a07b928b7d17f0edbad23c: start","job_status":"start","queue_duration_s":1.603406,"scheduling_latency_s":1.603433}1973gitlab # [ 426.632144] gitaly[934]: time="2026-10-03T07:06:26.486Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:56.484" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.485" grpc.time_ms=0.678 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab # [ 426.645012] gitaly[934]: time="2026-10-03T07:06:26.499Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-03T07:06:36.088" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:26.488" grpc.time_ms=10.2 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab # [ 426.754854] gitaly[934]: time="2026-10-03T07:06:26.654Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-03T07:06:56.653" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.653" grpc.time_ms=0.979 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 426.769273] gitaly[934]: time="2026-10-03T07:06:26.663Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=49 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-03T07:06:56.659" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:26.660" grpc.time_ms=3.073 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977gitlab # [ 426.882126] gitaly[934]: time="2026-10-03T07:06:26.782Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:36.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:26.771" grpc.time_ms=10.488 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1978gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'1979gitlab # [ 427.365606] nixbot[3609]: INFO: ::ffff:127.0.0.1:44204 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK1980gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.04 seconds)1981gitlab # [ 427.453360] gitaly[934]: time="2026-10-03T07:06:27.353Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=878 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-03T07:06:57.336" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:27.336" grpc.time_ms=16.918 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982gitlab # [ 427.478304] gitaly[934]: time="2026-10-03T07:06:27.373Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4098Y7CS57X4YGTS4JV543V grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-03T07:06:57.356" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:27.357" grpc.time_ms=15.839 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 427.512123] sidekiq-cluster[3482]: {"severity":"WARN","time":"2026-10-03T07:06:27.411Z","class":"CreatePipelineWorker","correlation_id":"01M4098Y7CS57X4YGTS4JV543V","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"a0a84e5503e4e6dbd339b9250ae18211495d317e","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"a0a84e5503e4e6dbd339b9250ae18211495d317e","ref":"refs/heads/master"},"retry":0}1984gitlab # [ 427.524876] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.421Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"60a07b928b7d17f0edbad23c","created_at":"2026-10-03T07:06:24.840Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ee28d6f97cac7facd39a4e2899b3a54ee1a15b32fb6fd96f32a97a8fbfbf5627","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.846Z","job_size_bytes":467,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-60a07b928b7d17f0edbad23c: done: 0.971715 sec","job_status":"done","queue_duration_s":1.603406,"scheduling_latency_s":1.603433,"gitaly_calls":7,"gitaly_duration_s":0.076032,"redis_calls":51,"redis_duration_s":0.010246,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.000934,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.004422,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002363,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.845668,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003984,"duration_s":0.971715,"completed_at":"2026-10-03T07:06:27.421Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053471,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1985gitlab # [ 427.591776] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.423Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"9bd0bf32-7c54-4647-b45c-773b4ff64fde\"}"],"class":"WebHookWorker","jid":"e4bb96348f5a67c6d4b4500b","created_at":"2026-10-03T07:06:24.989Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:231f9e6d09baa5bbedd8b3c4d7150ccafeeba45e7a5c5c86b1023ebbd5d614af","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.995Z","job_size_bytes":1751,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-e4bb96348f5a67c6d4b4500b: start","job_status":"start","queue_duration_s":2.428051,"scheduling_latency_s":2.428075}1986gitlab # [ 427.709274] nixbot[3609]: INFO: ::ffff:127.0.0.1:44212 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1987gitlab # [ 427.731045] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.630Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"9bd0bf32-7c54-4647-b45c-773b4ff64fde\"}"],"class":"WebHookWorker","jid":"e4bb96348f5a67c6d4b4500b","created_at":"2026-10-03T07:06:24.989Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-10c717d6eda743d2","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:231f9e6d09baa5bbedd8b3c4d7150ccafeeba45e7a5c5c86b1023ebbd5d614af","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:24.995Z","job_size_bytes":1751,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-e4bb96348f5a67c6d4b4500b: done: 0.206727 sec","job_status":"done","queue_duration_s":2.428051,"scheduling_latency_s":2.428075,"redis_calls":19,"redis_duration_s":0.004682,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001244,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000399,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002073,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06757499999997663,"cpu_s":0.124999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003306,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.206727,"completed_at":"2026-10-03T07:06:27.630Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1988gitlab # [ 427.804586] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.632Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c3ebb58fb2354d5957f0bd32","created_at":"2026-10-03T07:06:25.563Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.568Z","job_size_bytes":23,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c3ebb58fb2354d5957f0bd32: start","job_status":"start","queue_duration_s":2.064463,"scheduling_latency_s":2.064489}1989gitlab # [ 427.834083] gitaly[934]: time="2026-10-03T07:06:27.733Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BTY4CD12R9DTERTHZFE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:27.723" grpc.time_ms=10.852 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc1990gitlab # [ 427.848522] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:27 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 811991gitlab # [ 427.850699] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.737Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c3ebb58fb2354d5957f0bd32","created_at":"2026-10-03T07:06:25.563Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.568Z","job_size_bytes":23,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c3ebb58fb2354d5957f0bd32: done: 0.105114 sec","job_status":"done","queue_duration_s":2.064463,"scheduling_latency_s":2.064489,"redis_calls":10,"redis_duration_s":0.002564,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001568,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003152,"duration_s":0.105114,"completed_at":"2026-10-03T07:06:27.737Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017314,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1992gitlab # [ 427.896991] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.739Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a19c1930859f0f6393dc24c9","created_at":"2026-10-03T07:06:25.699Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.705Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a19c1930859f0f6393dc24c9: start","job_status":"start","queue_duration_s":2.033699,"scheduling_latency_s":2.033717}1993gitlab # [ 427.914526] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.784Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"a19c1930859f0f6393dc24c9","created_at":"2026-10-03T07:06:25.699Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.705Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-a19c1930859f0f6393dc24c9: done: 0.045063 sec","job_status":"done","queue_duration_s":2.033699,"scheduling_latency_s":2.033717,"redis_calls":10,"redis_duration_s":0.002508,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000836,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038018,"duration_s":0.045063,"completed_at":"2026-10-03T07:06:27.784Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1994gitlab # [ 427.962700] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.786Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"972c8885fce0078bc4cc81a6","created_at":"2026-10-03T07:06:25.723Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.730Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-972c8885fce0078bc4cc81a6: start","job_status":"start","queue_duration_s":2.055558,"scheduling_latency_s":2.055587}1995gitlab # [ 427.981227] gitaly[934]: time="2026-10-03T07:06:27.813Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M409BV0RHFPSE4PEF6T39MRT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:27.803" pid=934 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1996gitlab # [ 427.992335] gitlab-workhorse[820]: time="2026-10-03T07:06:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671997gitlab # [ 427.994305] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 771998gitlab # [ 427.996501] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.832Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"972c8885fce0078bc4cc81a6","created_at":"2026-10-03T07:06:25.723Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.730Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-972c8885fce0078bc4cc81a6: done: 0.046499 sec","job_status":"done","queue_duration_s":2.055558,"scheduling_latency_s":2.055587,"redis_calls":10,"redis_duration_s":0.001944,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00037,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031702,"duration_s":0.046499,"completed_at":"2026-10-03T07:06:27.832Z","load_balancing_strategy":"primary","db_duration_s":0.013546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1999gitlab # [ 428.043210] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.834Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"3f2180cfa205e8eeafcb5392","created_at":"2026-10-03T07:06:25.748Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.754Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-3f2180cfa205e8eeafcb5392: start","job_status":"start","queue_duration_s":2.08048,"scheduling_latency_s":2.080498}2000gitlab # [ 428.060409] gitaly[934]: time="2026-10-03T07:06:27.813Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M409BV0RHFPSE4PEF6T39MRT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:27.803" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2001gitlab # [ 428.076216] nixbot[3609]: {"ts": "2026-10-03T07:06:27.939Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}2002gitlab # [ 428.078965] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.875Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"3f2180cfa205e8eeafcb5392","created_at":"2026-10-03T07:06:25.748Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.754Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-3f2180cfa205e8eeafcb5392: done: 0.040924 sec","job_status":"done","queue_duration_s":2.08048,"scheduling_latency_s":2.080498,"redis_calls":13,"redis_duration_s":0.0037589999999999998,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000469,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001037,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001718,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028994,"duration_s":0.040924,"completed_at":"2026-10-03T07:06:27.875Z","load_balancing_strategy":"primary","db_duration_s":0.005704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab # [ 428.126752] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.877Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b023ac2c1b7d5210c4daba0c","created_at":"2026-10-03T07:06:25.754Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.760Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b023ac2c1b7d5210c4daba0c: start","job_status":"start","queue_duration_s":2.116851,"scheduling_latency_s":2.116954}2004gitlab # [ 428.144171] gitlab-workhorse[820]: time="2026-10-03T07:06:27Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3362005gitlab # [ 428.147123] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:27 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 822006gitlab # [ 428.149135] gitaly[934]: time="2026-10-03T07:06:27.813Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M409BV0RHFPSE4PEF6T39MRT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:27.803" grpc.time_ms=10.368 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2007gitlab # [ 428.164164] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.917Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"b023ac2c1b7d5210c4daba0c","created_at":"2026-10-03T07:06:25.754Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.760Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-b023ac2c1b7d5210c4daba0c: done: 0.04096 sec","job_status":"done","queue_duration_s":2.116851,"scheduling_latency_s":2.116954,"redis_calls":10,"redis_duration_s":0.002272,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000516,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000438,"duration_s":0.04096,"completed_at":"2026-10-03T07:06:27.917Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 428.210703] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.919Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"244268f98b4475b062b01046","created_at":"2026-10-03T07:06:25.760Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.767Z","job_size_bytes":19,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-244268f98b4475b062b01046: start","job_status":"start","queue_duration_s":2.151859,"scheduling_latency_s":2.15188}2009gitlab # [ 428.227947] gitaly[934]: time="2026-10-03T07:06:27.897Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M409BV384096TNYBA0MYX6H3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:27.877" pid=934 remote_ip="2001:db8:1::1" request_sha=a88b385eec447d48eacb46052fa1c8169a50f46f response_bytes=336 span.kind=server system=grpc2010gitlab # [ 428.238856] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.954Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"244268f98b4475b062b01046","created_at":"2026-10-03T07:06:25.760Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.767Z","job_size_bytes":19,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-244268f98b4475b062b01046: done: 0.034488 sec","job_status":"done","queue_duration_s":2.151859,"scheduling_latency_s":2.15188,"redis_calls":11,"redis_duration_s":0.001912,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000403,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001144,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022521,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025526,"duration_s":0.034488,"completed_at":"2026-10-03T07:06:27.953Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2011gitlab # [ 428.285068] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:27.955Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ac64b8a026fccf8f0d3aaa1f","created_at":"2026-10-03T07:06:25.770Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.775Z","job_size_bytes":66,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ac64b8a026fccf8f0d3aaa1f: start","job_status":"start","queue_duration_s":2.179826,"scheduling_latency_s":2.179897}2012gitlab # [ 428.303104] gitaly[934]: time="2026-10-03T07:06:27.897Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M409BV384096TNYBA0MYX6H3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:27.877" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2013gitlab # [ 428.317638] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.006Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ac64b8a026fccf8f0d3aaa1f","created_at":"2026-10-03T07:06:25.770Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.775Z","job_size_bytes":66,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ac64b8a026fccf8f0d3aaa1f: done: 0.051468 sec","job_status":"done","queue_duration_s":2.179826,"scheduling_latency_s":2.179897,"redis_calls":10,"redis_duration_s":0.0073279999999999994,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002469,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004313,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00142328,"duration_s":0.051468,"completed_at":"2026-10-03T07:06:28.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'2015gitlab # [ 428.365923] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.008Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"138f56973516c72cd5419144","created_at":"2026-10-03T07:06:25.776Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.781Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-138f56973516c72cd5419144: start","job_status":"start","queue_duration_s":2.227607,"scheduling_latency_s":2.227629}2016gitlab # [ 428.381843] gitaly[934]: time="2026-10-03T07:06:27.898Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M409BV384096TNYBA0MYX6H3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:27.877" grpc.time_ms=20.444 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2017gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.06 seconds)2018gitlab # [ 428.397491] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.054Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"138f56973516c72cd5419144","created_at":"2026-10-03T07:06:25.776Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.781Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-138f56973516c72cd5419144: done: 0.045797 sec","job_status":"done","queue_duration_s":2.227607,"scheduling_latency_s":2.227629,"redis_calls":10,"redis_duration_s":0.002102,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000441,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029408,"duration_s":0.045797,"completed_at":"2026-10-03T07:06:28.054Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2019gitlab # [ 428.443376] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.056Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1957c5c97386e1273946b70c","created_at":"2026-10-03T07:06:25.788Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.792Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1957c5c97386e1273946b70c: start","job_status":"start","queue_duration_s":2.263849,"scheduling_latency_s":2.263868}2020gitlab # [ 428.460690] gitaly[934]: time="2026-10-03T07:06:28.169Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:58.068" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.168" grpc.time_ms=0.736 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2021gitlab # [ 428.473965] nixbot[3609]: INFO: ::ffff:127.0.0.1:44220 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK2022gitlab # [ 428.475945] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.096Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"1957c5c97386e1273946b70c","created_at":"2026-10-03T07:06:25.788Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.792Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-1957c5c97386e1273946b70c: done: 0.040205 sec","job_status":"done","queue_duration_s":2.263849,"scheduling_latency_s":2.263868,"redis_calls":10,"redis_duration_s":0.001979,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000403,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029582,"duration_s":0.040205,"completed_at":"2026-10-03T07:06:28.096Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2023gitlab # [ 428.523048] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.098Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"254174c61a4a2f43d517ab61","created_at":"2026-10-03T07:06:25.792Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.797Z","job_size_bytes":6,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-254174c61a4a2f43d517ab61: start","job_status":"start","queue_duration_s":2.300928,"scheduling_latency_s":2.300947}2024gitlab # [ 428.539620] gitaly[934]: time="2026-10-03T07:06:28.319Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=7 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:38.203" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:28.303" grpc.time_ms=15.406 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025gitlab # [ 428.554043] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.123Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"254174c61a4a2f43d517ab61","created_at":"2026-10-03T07:06:25.792Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.797Z","job_size_bytes":6,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-254174c61a4a2f43d517ab61: done: 0.02564 sec","job_status":"done","queue_duration_s":2.300928,"scheduling_latency_s":2.300947,"redis_calls":10,"redis_duration_s":0.002054,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001204,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000435,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033094,"duration_s":0.02564,"completed_at":"2026-10-03T07:06:28.123Z","load_balancing_strategy":"primary","db_duration_s":0.002853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 428.600704] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"eb87c78edbf0733d59fe10ae","created_at":"2026-10-03T07:06:25.797Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.802Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-eb87c78edbf0733d59fe10ae: start","job_status":"start","queue_duration_s":2.322361,"scheduling_latency_s":2.322379}2027gitlab # [ 428.616783] gitaly[934]: time="2026-10-03T07:06:28.418Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ diskcache=2ae7c6de-5a65-49f2-9bd8-74b648637bb5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-03T13:06:28.406" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.407" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2028gitlab # [ 428.627091] gitaly[934]: time="2026-10-03T07:06:28.418Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-03T13:06:28.406" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.407" grpc.time_ms=10.962 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2029gitlab # [ 428.640879] nixbot[3609]: {"ts": "2026-10-03T07:06:28.470Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e \"HTTP/1.1 201 Created\""}2030gitlab # [ 428.643663] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.166Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"eb87c78edbf0733d59fe10ae","created_at":"2026-10-03T07:06:25.797Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.802Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-eb87c78edbf0733d59fe10ae: done: 0.041821 sec","job_status":"done","queue_duration_s":2.322361,"scheduling_latency_s":2.322379,"redis_calls":10,"redis_duration_s":0.0020099999999999996,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000339,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030356,"duration_s":0.041821,"completed_at":"2026-10-03T07:06:28.166Z","load_balancing_strategy":"primary","db_duration_s":0.010036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2031gitlab # [ 428.689541] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.168Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"26a17d09a304aaff63768ecc","created_at":"2026-10-03T07:06:25.809Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.814Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-26a17d09a304aaff63768ecc: start","job_status":"start","queue_duration_s":2.353962,"scheduling_latency_s":2.353983}2032gitlab # [ 428.706168] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 5162033gitlab # [ 428.708859] gitaly[934]: time="2026-10-03T07:06:28.472Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:58.471" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.472" grpc.time_ms=0.594 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034gitlab # [ 428.721917] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.374Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"26a17d09a304aaff63768ecc","created_at":"2026-10-03T07:06:25.809Z","trace_propagation_headers":{"sentry-trace":"73777835a82e497384dbeda257492698-0964b67e0dcf4a56","baggage":"sentry-trace_id=73777835a82e497384dbeda257492698,sentry-release=Unknown"},"correlation_id":"01M40993189A8N6MJSR9D8F3AY","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.814Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-26a17d09a304aaff63768ecc: done: 0.20609 sec","job_status":"done","queue_duration_s":2.353962,"scheduling_latency_s":2.353983,"redis_calls":14,"redis_duration_s":0.0045590000000000006,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000438,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001275,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00136108,"duration_s":0.20609,"completed_at":"2026-10-03T07:06:28.374Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2035gitlab # [ 428.768184] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.379Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2bf004b602186e5c35aecf37","created_at":"2026-10-03T07:06:25.978Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-c45d721aa5c549ed","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8f90c6ede3a627f0da910a80","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.980Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2bf004b602186e5c35aecf37: start","job_status":"start","queue_duration_s":2.398149,"scheduling_latency_s":2.398175}2036gitlab # [ 428.786080] gitaly[934]: time="2026-10-03T07:06:28.484Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-03T07:06:38.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:28.475" grpc.time_ms=9.342 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2037gitlab # [ 428.800995] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.440Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2bf004b602186e5c35aecf37","created_at":"2026-10-03T07:06:25.978Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-c45d721aa5c549ed","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8f90c6ede3a627f0da910a80","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:25.980Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2bf004b602186e5c35aecf37: done: 0.061904 sec","job_status":"done","queue_duration_s":2.398149,"scheduling_latency_s":2.398175,"gitaly_calls":1,"gitaly_duration_s":0.013499,"redis_calls":12,"redis_duration_s":0.007975,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.004069,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002518,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03746288,"duration_s":0.061904,"completed_at":"2026-10-03T07:06:28.440Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00985,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2038gitlab # [ 428.848487] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:28.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"022b8d7fb35b2d86c4ed5c31","created_at":"2026-10-03T07:06:26.113Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-c45d721aa5c549ed","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:46bcc95f8e5e38bea6dfb29df89bfbd41451baf1de6471e0abe1652a1b19b937","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:26.116Z","job_size_bytes":467,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-022b8d7fb35b2d86c4ed5c31: start","job_status":"start","queue_duration_s":2.327183,"scheduling_latency_s":2.327205}2039gitlab # [ 428.866599] gitaly[934]: time="2026-10-03T07:06:28.538Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-03T07:06:58.537" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.537" grpc.time_ms=0.964 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040gitlab # [ 428.880727] nixbot[3609]: {"ts": "2026-10-03T07:06:28.705Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e \"HTTP/1.1 201 Created\""}2041gitlab # [ 428.883908] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2212042gitlab # [ 428.886749] gitaly[934]: time="2026-10-03T07:06:28.549Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=237 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-03T07:06:58.542" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.543" grpc.time_ms=6.53 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2043gitlab # [ 428.903532] gitaly[934]: time="2026-10-03T07:06:28.574Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BVR2X9E2PZ2244ASD7KW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:58.572" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.573" grpc.time_ms=0.619 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2044gitlab # [ 428.917073] gitaly[934]: time="2026-10-03T07:06:28.588Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:38.107" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:28.577" grpc.time_ms=10.768 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 428.930737] gitaly[934]: time="2026-10-03T07:06:28.650Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BVR2X9E2PZ2244ASD7KW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:38.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:28.636" grpc.time_ms=13.217 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2046gitlab # [ 428.944795] gitaly[934]: time="2026-10-03T07:06:28.779Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BVZ8NNSRW4SARXHW9KXV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:58.677" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:28.778" grpc.time_ms=1.056 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047gitlab # [ 428.958237] gitaly[934]: time="2026-10-03T07:06:28.829Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=5 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BVZ8NNSRW4SARXHW9KXV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:38.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:28.818" grpc.time_ms=10.123 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2048gitlab # [ 428.995466] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1822049gitlab # [ 428.998391] nixbot[3609]: {"ts": "2026-10-03T07:06:28.897Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/a0a84e5503e4e6dbd339b9250ae18211495d317e \"HTTP/1.1 201 Created\""}2050gitlab # [ 429.113725] gitaly[934]: time="2026-10-03T07:06:29.013Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M409BW66124MF9HY1NZ3PRME grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.001" grpc.time_ms=11.879 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2051gitlab # [ 429.135609] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 792052gitlab # [ 429.208878] gitaly[934]: time="2026-10-03T07:06:29.109Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M409BW8RNYVBY5GDBCFF5242 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.095" pid=934 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2053gitlab # [ 429.225797] gitaly[934]: time="2026-10-03T07:06:29.109Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BW8RNYVBY5GDBCFF5242 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.095" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2054gitlab # [ 429.247421] gitlab-workhorse[820]: time="2026-10-03T07:06:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672055gitlab # [ 429.250431] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 942056gitlab # [ 429.253316] gitaly[934]: time="2026-10-03T07:06:29.109Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BW8RNYVBY5GDBCFF5242 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.095" grpc.time_ms=13.68 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2057gitlab # [ 429.272530] gitaly[934]: time="2026-10-03T07:06:29.124Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-03T07:06:59.105" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:29.106" grpc.time_ms=18.57 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058gitlab # [ 429.289768] sidekiq-cluster[3482]: {"severity":"WARN","time":"2026-10-03T07:06:29.165Z","class":"CreatePipelineWorker","correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"a0a84e5503e4e6dbd339b9250ae18211495d317e","after":"e874471c6bba3c76d684eaabbff376e5652e08d0","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"e874471c6bba3c76d684eaabbff376e5652e08d0","ref":"refs/heads/master"},"retry":0}2059gitlab # [ 429.297609] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.172Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"022b8d7fb35b2d86c4ed5c31","created_at":"2026-10-03T07:06:26.113Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-c45d721aa5c549ed","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:46bcc95f8e5e38bea6dfb29df89bfbd41451baf1de6471e0abe1652a1b19b937","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:26.116Z","job_size_bytes":467,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-022b8d7fb35b2d86c4ed5c31: done: 0.728641 sec","job_status":"done","queue_duration_s":2.327183,"scheduling_latency_s":2.327205,"gitaly_calls":7,"gitaly_duration_s":0.082857,"redis_calls":33,"redis_duration_s":0.006602,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000776,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002079,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.607379,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.728641,"completed_at":"2026-10-03T07:06:29.171Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032755,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2060gitlab # [ 429.347265] gitaly[934]: time="2026-10-03T07:06:29.147Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BMHXS0AYFYBQ7XX2TTVZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-03T07:06:59.128" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:29.128" grpc.time_ms=18.096 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061gitlab # [ 429.361422] nixbot[3609]: INFO: ::ffff:127.0.0.1:44228 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2062gitlab # [ 429.362763] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.175Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"828d17a5-4e04-42a2-8732-c030bb8a7c95\"}"],"class":"WebHookWorker","jid":"eaac0b93a7b06f316e780601","created_at":"2026-10-03T07:06:26.131Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-c45d721aa5c549ed","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cc341aaf6454d2c8201d4180da98b2e6fea9ac25c76d85217fa0548a632c6c04","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:26.133Z","job_size_bytes":1749,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-eaac0b93a7b06f316e780601: start","job_status":"start","queue_duration_s":3.041663,"scheduling_latency_s":3.041693}2063gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'2064gitlab # [ 429.381103] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.218Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"828d17a5-4e04-42a2-8732-c030bb8a7c95\"}"],"class":"WebHookWorker","jid":"eaac0b93a7b06f316e780601","created_at":"2026-10-03T07:06:26.131Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-c45d721aa5c549ed","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cc341aaf6454d2c8201d4180da98b2e6fea9ac25c76d85217fa0548a632c6c04","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:26.133Z","job_size_bytes":1749,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-eaac0b93a7b06f316e780601: done: 0.043462 sec","job_status":"done","queue_duration_s":3.041663,"scheduling_latency_s":3.041693,"redis_calls":14,"redis_duration_s":0.002037,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000262,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001051,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000724,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.006943239999998241,"cpu_s":0.027953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.043462,"completed_at":"2026-10-03T07:06:29.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 429.431332] gitaly[934]: time="2026-10-03T07:06:29.323Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M409BWFPFS9PDEDWAYGNSCSY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.307" grpc.time_ms=16.259 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2066gitlab # [ 429.444169] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 852067gitlab # [ 429.446380] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.220Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6880d75e50a34e32f6a8c9aa","created_at":"2026-10-03T07:06:26.266Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-9b1a0d4ba34242fc","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:26.272Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6880d75e50a34e32f6a8c9aa: start","job_status":"start","queue_duration_s":2.947711,"scheduling_latency_s":2.947733}2068gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.06 seconds)2069gitlab # [ 429.463571] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.275Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"6880d75e50a34e32f6a8c9aa","created_at":"2026-10-03T07:06:26.266Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-9b1a0d4ba34242fc","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:26.272Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-6880d75e50a34e32f6a8c9aa: done: 0.055698 sec","job_status":"done","queue_duration_s":2.947711,"scheduling_latency_s":2.947733,"redis_calls":19,"redis_duration_s":0.003102,"redis_read_bytes":21,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00067,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000466,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001395,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000571,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00862796,"duration_s":0.055698,"completed_at":"2026-10-03T07:06:29.275Z","load_balancing_strategy":"primary","db_duration_s":0.0084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 429.511875] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.278Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","e4bb96348f5a67c6d4b4500b"],"class":"WebHooks::LogExecutionWorker","jid":"5c5be7ea9f5d59c18d8ea2f3","created_at":"2026-10-03T07:06:27.612Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-4a114916c1df4ca8","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d2c038200ced4e7321be0030f850883c479dcc32bd6273985300bc54fde64872","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:27.620Z","job_size_bytes":2593,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5c5be7ea9f5d59c18d8ea2f3: start","job_status":"start","queue_duration_s":1.658889,"scheduling_latency_s":1.658911}2071gitlab # [ 429.530122] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.331Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","e4bb96348f5a67c6d4b4500b"],"class":"WebHooks::LogExecutionWorker","jid":"5c5be7ea9f5d59c18d8ea2f3","created_at":"2026-10-03T07:06:27.612Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-4a114916c1df4ca8","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d2c038200ced4e7321be0030f850883c479dcc32bd6273985300bc54fde64872","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:27.620Z","job_size_bytes":2593,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-5c5be7ea9f5d59c18d8ea2f3: done: 0.052391 sec","job_status":"done","queue_duration_s":1.658889,"scheduling_latency_s":1.658911,"redis_calls":10,"redis_duration_s":0.00199,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000668,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023748,"duration_s":0.052391,"completed_at":"2026-10-03T07:06:29.331Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013415,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072gitlab # [ 429.576778] nixbot[3609]: INFO: ::ffff:127.0.0.1:44238 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK2073gitlab # [ 429.578912] gitlab-workhorse[820]: time="2026-10-03T07:06:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672074gitlab # [ 429.581055] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1292075gitlab # [ 429.583107] gitaly[934]: time="2026-10-03T07:06:29.454Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M409BWJDGFKG9EGM02D2SDKA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.441" pid=934 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2076gitlab # [ 429.594618] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.332Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"b9569b62be31c61b5a75f1ff","created_at":"2026-10-03T07:06:28.438Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-2a47bd77b7164550","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:28.445Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b9569b62be31c61b5a75f1ff: start","job_status":"start","queue_duration_s":0.887785,"scheduling_latency_s":0.887808}2077gitlab # [ 429.611455] gitaly[934]: time="2026-10-03T07:06:29.454Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWJDGFKG9EGM02D2SDKA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.441" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2078gitlab # [ 429.626425] gitaly[934]: time="2026-10-03T07:06:29.455Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWJDGFKG9EGM02D2SDKA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.441" grpc.time_ms=13.286 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2079gitlab # [ 429.641773] gitaly[934]: time="2026-10-03T07:06:29.469Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:29.469" grpc.time_ms=0.578 pid=934 span.kind=server system=grpc2080gitlab # [ 429.652242] gitaly[934]: time="2026-10-03T07:06:29.506Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:29.505" grpc.time_ms=0.573 pid=934 span.kind=server system=grpc2081gitlab # [ 429.662174] gitaly[934]: time="2026-10-03T07:06:29.539Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:29.538" grpc.time_ms=0.519 pid=934 span.kind=server system=grpc2082gitlab # [ 429.672366] gitaly[934]: time="2026-10-03T07:06:29.547Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 diskcache=ff0eb1e6-3744-4ff7-8b0b-a1d0260b0b46 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:06:59.418" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:29.419" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2083gitlab # [ 429.682546] gitaly[934]: time="2026-10-03T07:06:29.547Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=114 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7372 command.oublock=0 command.real_time_ms=123 command.spawn_token_fork_ms=0 command.system_time_ms=73 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:06:59.418" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:29.419" grpc.time_ms=127.772 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2084gitlab # [ 429.696064] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:29 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 792085gitlab # [ 429.698193] gitaly[934]: time="2026-10-03T07:06:29.576Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BWQRMW149P6Q2YAAYR8N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.565" grpc.time_ms=10.624 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2086gitlab # [ 429.733680] gitaly[934]: time="2026-10-03T07:06:29.634Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-03T07:06:59.632" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:29.632" grpc.time_ms=1.104 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2087gitlab # [ 429.769060] gitaly[934]: time="2026-10-03T07:06:29.669Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M409BWTBTQ95JVVKNRDFVEPC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.646" pid=934 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2088gitlab # [ 429.787410] gitaly[934]: time="2026-10-03T07:06:29.669Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWTBTQ95JVVKNRDFVEPC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.646" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2089gitlab # [ 429.807878] gitlab-workhorse[820]: time="2026-10-03T07:06:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162090gitlab # [ 429.810585] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:29 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 902091gitlab # [ 429.813054] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.697Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"b9569b62be31c61b5a75f1ff","created_at":"2026-10-03T07:06:28.438Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-2a47bd77b7164550","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:28.445Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3482,"job_status":"rescheduled","message":"PipelineProcessWorker JID-b9569b62be31c61b5a75f1ff: rescheduled"}2092gitlab # [ 429.830890] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.706Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"b9569b62be31c61b5a75f1ff","created_at":"2026-10-03T07:06:28.438Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-2a47bd77b7164550","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:28.445Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b9569b62be31c61b5a75f1ff: done: 0.373558 sec","job_status":"done","queue_duration_s":0.887785,"scheduling_latency_s":0.887808,"gitaly_calls":3,"gitaly_duration_s":0.137741,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.031016,"redis_read_bytes":148,"redis_write_bytes":18012,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001502,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000821,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.007591,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12311,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.016322,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00037,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000519,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002389,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.174083,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.32315188,"duration_s":0.373558,"completed_at":"2026-10-03T07:06:29.706Z","load_balancing_strategy":"primary","db_duration_s":0.042157,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2093gitlab # [ 429.889563] gitaly[934]: time="2026-10-03T07:06:29.669Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWTBTQ95JVVKNRDFVEPC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:29.646" grpc.time_ms=23.452 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2094gitlab # [ 429.905529] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.708Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","eaac0b93a7b06f316e780601"],"class":"WebHooks::LogExecutionWorker","jid":"d78941078bc2929887643aff","created_at":"2026-10-03T07:06:29.210Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-3ff11adf56754f1e","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5a8041eee5285c29e6c394e42f8f927a0c365d09187ffd127530fdc691b73f12","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.212Z","job_size_bytes":2592,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-d78941078bc2929887643aff: start","job_status":"start","queue_duration_s":0.495757,"scheduling_latency_s":0.495783}2095gitlab # [ 429.923066] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.746Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","eaac0b93a7b06f316e780601"],"class":"WebHooks::LogExecutionWorker","jid":"d78941078bc2929887643aff","created_at":"2026-10-03T07:06:29.210Z","trace_propagation_headers":{"sentry-trace":"c0d571441c354d568ddf6b1067d7a979-3ff11adf56754f1e","baggage":"sentry-trace_id=c0d571441c354d568ddf6b1067d7a979,sentry-release=Unknown"},"correlation_id":"01M409BMHXS0AYFYBQ7XX2TTVZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5a8041eee5285c29e6c394e42f8f927a0c365d09187ffd127530fdc691b73f12","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.212Z","job_size_bytes":2592,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-d78941078bc2929887643aff: done: 0.037806 sec","job_status":"done","queue_duration_s":0.495757,"scheduling_latency_s":0.495783,"redis_calls":7,"redis_duration_s":0.003123,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002574,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000549,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026149,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037806,"completed_at":"2026-10-03T07:06:29.746Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00405,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2096gitlab # [ 429.968973] gitaly[934]: time="2026-10-03T07:06:29.682Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BV7GZ1JC5BBTF21X4J98 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:59.680" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:29.681" grpc.time_ms=0.752 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2097gitlab # [ 429.981834] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.748Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"011eea977354d70eb262a7f6","created_at":"2026-10-03T07:06:29.261Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-30a454404f3d4d9d","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.266Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-011eea977354d70eb262a7f6: start","job_status":"start","queue_duration_s":0.482043,"scheduling_latency_s":0.482074}2098gitlab # [ 429.999610] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.861Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"011eea977354d70eb262a7f6","created_at":"2026-10-03T07:06:29.261Z","trace_propagation_headers":{"sentry-trace":"4840591f6de44ab083cbbeb96adfea25-30a454404f3d4d9d","baggage":"sentry-trace_id=4840591f6de44ab083cbbeb96adfea25,sentry-release=Unknown"},"correlation_id":"01M4098Y7CS57X4YGTS4JV543V","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.266Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-011eea977354d70eb262a7f6: done: 0.112291 sec","job_status":"done","queue_duration_s":0.482043,"scheduling_latency_s":0.482074,"redis_calls":11,"redis_duration_s":0.002513,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000441,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001557,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003181,"duration_s":0.112291,"completed_at":"2026-10-03T07:06:29.861Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 430.048096] gitaly[934]: time="2026-10-03T07:06:29.801Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:59.798" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:29.799" grpc.time_ms=1.529 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2100gitlab # [ 430.061831] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.862Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a0fa60b253dd04511b8d3824","created_at":"2026-10-03T07:06:29.409Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.412Z","job_size_bytes":23,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a0fa60b253dd04511b8d3824: start","job_status":"start","queue_duration_s":0.45034,"scheduling_latency_s":0.450372}2101gitlab # [ 430.079289] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.915Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a0fa60b253dd04511b8d3824","created_at":"2026-10-03T07:06:29.409Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.412Z","job_size_bytes":23,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a0fa60b253dd04511b8d3824: done: 0.052838 sec","job_status":"done","queue_duration_s":0.45034,"scheduling_latency_s":0.450372,"redis_calls":7,"redis_duration_s":0.002202,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000403,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052838,"completed_at":"2026-10-03T07:06:29.915Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013863,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2102gitlab # [ 430.124464] gitaly[934]: time="2026-10-03T07:06:29.827Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-03T07:06:59.815" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:29.815" grpc.time_ms=11.946 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2103gitlab # [ 430.138892] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.916Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5d09fd3d6599bc73f8f0565d","created_at":"2026-10-03T07:06:29.553Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.556Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5d09fd3d6599bc73f8f0565d: start","job_status":"start","queue_duration_s":0.360091,"scheduling_latency_s":0.36011}2104gitlab # [ 430.156589] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.947Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5d09fd3d6599bc73f8f0565d","created_at":"2026-10-03T07:06:29.553Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.556Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5d09fd3d6599bc73f8f0565d: done: 0.031391 sec","job_status":"done","queue_duration_s":0.360091,"scheduling_latency_s":0.36011,"redis_calls":7,"redis_duration_s":0.001358,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000355,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031391,"completed_at":"2026-10-03T07:06:29.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2105gitlab # [ 430.201280] gitaly[934]: time="2026-10-03T07:06:29.846Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:39.205" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:29.835" grpc.time_ms=11.05 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2106gitlab # [ 430.215750] nixbot[3609]: {"ts": "2026-10-03T07:06:30.082Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 \"HTTP/1.1 201 Created\""}2107gitlab # [ 430.218818] nixbot[3609]: {"ts": "2026-10-03T07:06:30.094Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-e874471c6bba-5"}2108gitlab # [ 430.221356] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.951Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"45bb6ba31f85f4c03304a62c","created_at":"2026-10-03T07:06:29.567Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.570Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-45bb6ba31f85f4c03304a62c: start","job_status":"start","queue_duration_s":0.380135,"scheduling_latency_s":0.380155}2109gitlab # [ 430.237979] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.984Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"45bb6ba31f85f4c03304a62c","created_at":"2026-10-03T07:06:29.567Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.570Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-45bb6ba31f85f4c03304a62c: done: 0.03393 sec","job_status":"done","queue_duration_s":0.380135,"scheduling_latency_s":0.380155,"redis_calls":7,"redis_duration_s":0.0033840000000000003,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002612,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03393,"completed_at":"2026-10-03T07:06:29.984Z","load_balancing_strategy":"primary","db_duration_s":0.006251,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2110gitlab # [ 430.283287] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3562111gitlab # [ 430.286089] gitaly[934]: time="2026-10-03T07:06:29.869Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-03T07:06:39.238" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:29.869" grpc.time_ms=0.427 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab # [ 430.297601] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:29.986Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"291d258f81bdefb844fa0527","created_at":"2026-10-03T07:06:29.580Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.585Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-291d258f81bdefb844fa0527: start","job_status":"start","queue_duration_s":0.400513,"scheduling_latency_s":0.400531}2113gitlab # [ 430.314046] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.018Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"291d258f81bdefb844fa0527","created_at":"2026-10-03T07:06:29.580Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.585Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-291d258f81bdefb844fa0527: done: 0.032417 sec","job_status":"done","queue_duration_s":0.400513,"scheduling_latency_s":0.400531,"redis_calls":10,"redis_duration_s":0.0018189999999999999,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000444,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022517,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032417,"completed_at":"2026-10-03T07:06:30.018Z","load_balancing_strategy":"primary","db_duration_s":0.005806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2114gitlab # [ 430.359507] gitaly[934]: time="2026-10-03T07:06:29.887Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-03T07:06:39.244" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:29.875" grpc.time_ms=12.403 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2115gitlab # [ 430.373774] gitaly[934]: time="2026-10-03T07:06:29.941Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:29.939" grpc.time_ms=1.564 pid=934 span.kind=server system=grpc2116gitlab # [ 430.383709] nixbot[3609]: {"ts": "2026-10-03T07:06:30.258Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-e874471c6bba-5", "argv0": "bwrap"}2117gitlab # [ 430.386348] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.020Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bf156dd56a73c4ed815beb49","created_at":"2026-10-03T07:06:29.586Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.590Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bf156dd56a73c4ed815beb49: start","job_status":"start","queue_duration_s":0.430519,"scheduling_latency_s":0.430538}2118gitlab # [ 430.404230] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.047Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bf156dd56a73c4ed815beb49","created_at":"2026-10-03T07:06:29.586Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.590Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bf156dd56a73c4ed815beb49: done: 0.02711 sec","job_status":"done","queue_duration_s":0.430519,"scheduling_latency_s":0.430538,"redis_calls":7,"redis_duration_s":0.0014140000000000001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02711,"completed_at":"2026-10-03T07:06:30.047Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119gitlab # [ 430.449574] gitaly[934]: time="2026-10-03T07:06:29.978Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:29.977" grpc.time_ms=0.458 pid=934 span.kind=server system=grpc2120gitlab # [ 430.459645] gitaly[934]: time="2026-10-03T07:06:29.985Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM diskcache=c889175e-9550-4c14-819a-da0a4bba8a6b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-03T07:06:39.260" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:29.890" pid=934 ref=refs/keep-around/e874471c6bba3c76d684eaabbff376e5652e08d0 remote_ip="2001:db8:1::1" revision=e874471c6bba3c76d684eaabbff376e5652e08d0 span.kind=server system=grpc username=root2121gitlab # [ 430.471947] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.049Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"362c8f1d9a3c9a80a0bf2f46","created_at":"2026-10-03T07:06:29.591Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.596Z","job_size_bytes":19,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-362c8f1d9a3c9a80a0bf2f46: start","job_status":"start","queue_duration_s":0.452613,"scheduling_latency_s":0.452631}2122gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0'2123gitlab # [ 430.490198] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.076Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"362c8f1d9a3c9a80a0bf2f46","created_at":"2026-10-03T07:06:29.591Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.596Z","job_size_bytes":19,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-362c8f1d9a3c9a80a0bf2f46: done: 0.027113 sec","job_status":"done","queue_duration_s":0.452613,"scheduling_latency_s":0.452631,"redis_calls":8,"redis_duration_s":0.002476,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027113,"completed_at":"2026-10-03T07:06:30.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 430.533939] gitaly[934]: time="2026-10-03T07:06:29.986Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=86 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5287 command.oublock=8 command.real_time_ms=90 command.spawn_token_fork_ms=1 command.system_time_ms=40 command.user_time_ms=46 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-03T07:06:39.260" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-03T07:06:29.890" grpc.time_ms=95.418 pid=934 ref=refs/keep-around/e874471c6bba3c76d684eaabbff376e5652e08d0 remote_ip="2001:db8:1::1" revision=e874471c6bba3c76d684eaabbff376e5652e08d0 span.kind=server system=grpc username=root2125gitlab # [ 430.549704] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.077Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2b16b84460b535872eecd0ae","created_at":"2026-10-03T07:06:29.597Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.601Z","job_size_bytes":66,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2b16b84460b535872eecd0ae: start","job_status":"start","queue_duration_s":0.475667,"scheduling_latency_s":0.475685}2126gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0', in 0.09 seconds)2127(finished: subtest: push delivers the webhook, in 93.87 seconds)2128subtest: eval and build post commit statuses2129gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2130gitlab # [ 430.568373] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.102Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2b16b84460b535872eecd0ae","created_at":"2026-10-03T07:06:29.597Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.601Z","job_size_bytes":66,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2b16b84460b535872eecd0ae: done: 0.025048 sec","job_status":"done","queue_duration_s":0.475667,"scheduling_latency_s":0.475685,"redis_calls":7,"redis_duration_s":0.001362,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025048,"completed_at":"2026-10-03T07:06:30.102Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131gitlab # [ 430.614365] gitaly[934]: time="2026-10-03T07:06:29.989Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:06:59.088" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:29.989" grpc.time_ms=0.462 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2132gitlab # [ 430.627857] nixbot[3609]: INFO: ::ffff:127.0.0.1:44248 - "GET /api/repos/gitlab/root/test-flake/builds?commit=e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 200 OK2133gitlab # [ 430.630112] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.103Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a6fd8eaa40789ce4bb96a94c","created_at":"2026-10-03T07:06:29.602Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.606Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a6fd8eaa40789ce4bb96a94c: start","job_status":"start","queue_duration_s":0.497698,"scheduling_latency_s":0.497715}2134gitlab # [ 430.647282] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.132Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a6fd8eaa40789ce4bb96a94c","created_at":"2026-10-03T07:06:29.602Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.606Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a6fd8eaa40789ce4bb96a94c: done: 0.029055 sec","job_status":"done","queue_duration_s":0.497698,"scheduling_latency_s":0.497715,"redis_calls":7,"redis_duration_s":0.001599,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001165,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029055,"completed_at":"2026-10-03T07:06:30.132Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2135gitlab # [ 430.693838] gitaly[934]: time="2026-10-03T07:06:30.381Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-03T07:07:00.379" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:30.379" grpc.time_ms=1.09 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2136gitlab # [ 430.707677] nix-daemon[3765]: accepted connection from pid 4855, user nixbot2137gitlab # [ 430.708765] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.134Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"21f443014db053b6712e61f1","created_at":"2026-10-03T07:06:29.607Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.611Z","job_size_bytes":6,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-21f443014db053b6712e61f1: start","job_status":"start","queue_duration_s":0.522786,"scheduling_latency_s":0.522804}2138gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.17 seconds)2139gitlab # [ 430.725428] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.148Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"21f443014db053b6712e61f1","created_at":"2026-10-03T07:06:29.607Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.611Z","job_size_bytes":6,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-21f443014db053b6712e61f1: done: 0.014569 sec","job_status":"done","queue_duration_s":0.522786,"scheduling_latency_s":0.522804,"redis_calls":7,"redis_duration_s":0.001413,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009369,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014569,"completed_at":"2026-10-03T07:06:30.148Z","load_balancing_strategy":"primary","db_duration_s":0.000525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab # [ 430.769287] gitaly[934]: time="2026-10-03T07:06:30.448Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BWYVTZ3AVDZMYTQJXEWM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:00.443" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:30.444" grpc.time_ms=4.431 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2141gitlab # [ 430.782103] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1012142gitlab # [ 430.784877] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.150Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"92ae6f891b4541c93ed152ab","created_at":"2026-10-03T07:06:29.612Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.615Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-92ae6f891b4541c93ed152ab: start","job_status":"start","queue_duration_s":0.534265,"scheduling_latency_s":0.534281}2143gitlab # [ 430.800606] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"92ae6f891b4541c93ed152ab","created_at":"2026-10-03T07:06:29.612Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.615Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-92ae6f891b4541c93ed152ab: done: 0.030275 sec","job_status":"done","queue_duration_s":0.534265,"scheduling_latency_s":0.534281,"redis_calls":7,"redis_duration_s":0.001332,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030275,"completed_at":"2026-10-03T07:06:30.180Z","load_balancing_strategy":"primary","db_duration_s":0.009326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2144gitlab # [ 430.843791] gitaly[934]: time="2026-10-03T07:06:30.605Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BXRCK9VGMAJNYBZNYQZS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:00.604" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:30.604" grpc.time_ms=0.667 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2145gitlab # [ 430.857041] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.181Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8a22fcd63c34e0118d456690","created_at":"2026-10-03T07:06:29.698Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.700Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8a22fcd63c34e0118d456690: start","job_status":"start","queue_duration_s":0.481225,"scheduling_latency_s":0.481243}2146gitlab # [ 430.873235] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.211Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"8a22fcd63c34e0118d456690","created_at":"2026-10-03T07:06:29.698Z","trace_propagation_headers":{"sentry-trace":"642e55722d644e1da36064bec76df7c2-958d2865c2e24f04","baggage":"sentry-trace_id=642e55722d644e1da36064bec76df7c2,sentry-release=Unknown"},"correlation_id":"01M409BV7GZ1JC5BBTF21X4J98","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.700Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8a22fcd63c34e0118d456690: done: 0.029559 sec","job_status":"done","queue_duration_s":0.481225,"scheduling_latency_s":0.481243,"redis_calls":8,"redis_duration_s":0.001632,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029559,"completed_at":"2026-10-03T07:06:30.211Z","load_balancing_strategy":"primary","db_duration_s":0.004649,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2147gitlab # [ 430.917856] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.212Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"18bccd0464b6f3f98dfc98fb","created_at":"2026-10-03T07:06:29.991Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-78198cde965849b2","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.997Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-18bccd0464b6f3f98dfc98fb: start","job_status":"start","queue_duration_s":0.215202,"scheduling_latency_s":0.215221}2148gitlab # [ 430.935350] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.243Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"18bccd0464b6f3f98dfc98fb","created_at":"2026-10-03T07:06:29.991Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-78198cde965849b2","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:29.997Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-18bccd0464b6f3f98dfc98fb: done: 0.030947 sec","job_status":"done","queue_duration_s":0.215202,"scheduling_latency_s":0.215221,"redis_calls":7,"redis_duration_s":0.001457,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001068,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030947,"completed_at":"2026-10-03T07:06:30.243Z","load_balancing_strategy":"primary","db_duration_s":0.005758,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2149gitlab # [ 430.980326] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.244Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b23fc4841ef87333cfa05994","created_at":"2026-10-03T07:06:29.998Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-78198cde965849b2","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:30.002Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b23fc4841ef87333cfa05994: start","job_status":"start","queue_duration_s":0.24254,"scheduling_latency_s":0.24256}2150gitlab # [ 430.998831] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.301Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b23fc4841ef87333cfa05994","created_at":"2026-10-03T07:06:29.998Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-78198cde965849b2","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:30.002Z","job_size_bytes":65,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b23fc4841ef87333cfa05994: done: 0.056658 sec","job_status":"done","queue_duration_s":0.24254,"scheduling_latency_s":0.24256,"redis_calls":10,"redis_duration_s":0.002168,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001092,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.056658,"completed_at":"2026-10-03T07:06:30.301Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017877,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151gitlab # [ 431.045617] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.302Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b7509f6be2aed0b94ba46c30","created_at":"2026-10-03T07:06:30.058Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-78198cde965849b2","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:30.062Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b7509f6be2aed0b94ba46c30: start","job_status":"start","queue_duration_s":0.239967,"scheduling_latency_s":0.239987}2152gitlab # [ 431.062828] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.480Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b7509f6be2aed0b94ba46c30","created_at":"2026-10-03T07:06:30.058Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-78198cde965849b2","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:30.062Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-b7509f6be2aed0b94ba46c30: done: 0.17749 sec","job_status":"done","queue_duration_s":0.239967,"scheduling_latency_s":0.239987,"gitaly_calls":2,"gitaly_duration_s":0.015703,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.018512,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001053,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002935,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000346,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002771,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000238,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.008957,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001159,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118726,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13500156,"duration_s":0.17749,"completed_at":"2026-10-03T07:06:30.480Z","load_balancing_strategy":"primary","db_duration_s":0.035304,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2153gitlab # [ 431.118437] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b62c33f4bafba0162a8f9886","created_at":"2026-10-03T07:06:30.361Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-c5e75cfc41244bc6","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:30.363Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b62c33f4bafba0162a8f9886: start","job_status":"start","queue_duration_s":0.118963,"scheduling_latency_s":0.118989}2154gitlab # [ 431.136113] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:30.547Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"b62c33f4bafba0162a8f9886","created_at":"2026-10-03T07:06:30.361Z","trace_propagation_headers":{"sentry-trace":"f6e8d837d9d14ce4808d8579231d7d25-c5e75cfc41244bc6","baggage":"sentry-trace_id=f6e8d837d9d14ce4808d8579231d7d25,sentry-release=Unknown"},"correlation_id":"01M409BWYVTZ3AVDZMYTQJXEWM","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:30.363Z","job_size_bytes":3,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-b62c33f4bafba0162a8f9886: done: 0.065316 sec","job_status":"done","queue_duration_s":0.118963,"scheduling_latency_s":0.118989,"redis_calls":7,"redis_duration_s":0.0017079999999999999,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065316,"completed_at":"2026-10-03T07:06:30.547Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2155gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2156gitlab # [ 431.853487] gitaly[934]: time="2026-10-03T07:06:31.753Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BYW8ZGXCD9V0H54CB5M7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:01.751" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:31.752" grpc.time_ms=0.866 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2157gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.14 seconds)2158gitlab # [ 431.886309] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 972159gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2160gitlab # [ 432.991063] gitaly[934]: time="2026-10-03T07:06:32.890Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409BZZN502GJT4GAR2ZW5Z6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:02.889" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:32.889" grpc.time_ms=0.846 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.14 seconds)2162gitlab # [ 433.025399] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1022163gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2164gitlab # [ 434.139509] gitaly[934]: time="2026-10-03T07:06:34.039Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C13ASWNE8BDEE1S52ZES grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:04.037" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:34.038" grpc.time_ms=0.893 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2165gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.15 seconds)2166gitlab # [ 434.180357] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1172167gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2168gitlab # [ 435.284309] gitaly[934]: time="2026-10-03T07:06:35.183Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C27C0GA0260TGEX5RV5W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:05.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:35.182" grpc.time_ms=0.929 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2169gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.14 seconds)2170gitlab # [ 435.318425] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1012171gitlab # [ 435.673941] nix-daemon[3765]: accepted connection from pid 4896, user nixbot2172gitlab # [ 435.687198] nix-daemon[3765]: accepted connection from pid 4855, user nixbot2173gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2174gitlab # [ 436.321495] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:36.220Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"4348987bb483fc0297473cf7","created_at":"2026-10-03T07:06:36.168Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a455b11a4776f25eba04d07172ceaf8d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.218Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-4348987bb483fc0297473cf7: start","job_status":"start","queue_duration_s":0.002289,"scheduling_latency_s":0.002326}2175gitlab # [ 436.452155] gitaly[934]: time="2026-10-03T07:06:36.351Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C3BNC9KVQDJSXGMEE8NR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:06.250" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:36.350" grpc.time_ms=0.896 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2176gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.17 seconds)2177gitlab # [ 436.472547] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:36.366Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"4348987bb483fc0297473cf7","created_at":"2026-10-03T07:06:36.168Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a455b11a4776f25eba04d07172ceaf8d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.218Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-4348987bb483fc0297473cf7: done: 0.145952 sec","job_status":"done","queue_duration_s":0.002289,"scheduling_latency_s":0.002326,"redis_calls":9,"redis_duration_s":0.013282,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003162,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004609,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005511,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.145952,"completed_at":"2026-10-03T07:06:36.366Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2178gitlab # [ 436.525066] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:36.377Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cd128a2a8f2834ac9f9eb608","created_at":"2026-10-03T07:06:36.228Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9fbf239f4bfe18cb35c918718068a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.238Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cd128a2a8f2834ac9f9eb608: start","job_status":"start","queue_duration_s":0.138656,"scheduling_latency_s":0.13869}2179gitlab # [ 436.540707] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1072180gitlab # [ 437.410914] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cd128a2a8f2834ac9f9eb608","created_at":"2026-10-03T07:06:36.228Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9fbf239f4bfe18cb35c918718068a0f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.238Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cd128a2a8f2834ac9f9eb608: done: 0.932817 sec","job_status":"done","queue_duration_s":0.138656,"scheduling_latency_s":0.13869,"redis_calls":564,"redis_duration_s":0.5194529999999999,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001871,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.515373,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.42347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.932817,"completed_at":"2026-10-03T07:06:37.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2181gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2182gitlab # [ 437.422389] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.311Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9fc63653c6706568832aa58e","created_at":"2026-10-03T07:06:36.244Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95a4f10f3d62a7b7ed26b7248e4f9fbb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.268Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9fc63653c6706568832aa58e: start","job_status":"start","queue_duration_s":1.043598,"scheduling_latency_s":1.043619}2183gitlab # [ 437.501671] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"9fc63653c6706568832aa58e","created_at":"2026-10-03T07:06:36.244Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95a4f10f3d62a7b7ed26b7248e4f9fbb","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.268Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-9fc63653c6706568832aa58e: done: 0.030407 sec","job_status":"done","queue_duration_s":1.043598,"scheduling_latency_s":1.043619,"redis_calls":11,"redis_duration_s":0.002833,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000428,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00129,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005897,"exclusive_lock_hold_duration_s":0.00577286,"duration_s":0.030407,"completed_at":"2026-10-03T07:06:37.341Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184gitlab # [ 437.558004] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e81411fae27a7644109042f2","created_at":"2026-10-03T07:06:36.273Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f8fe67fe1c14a7b123507679e81d75c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.286Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e81411fae27a7644109042f2: start","job_status":"start","queue_duration_s":1.057091,"scheduling_latency_s":1.057111}2185gitlab # [ 437.572748] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"e81411fae27a7644109042f2","created_at":"2026-10-03T07:06:36.273Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f8fe67fe1c14a7b123507679e81d75c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.286Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-e81411fae27a7644109042f2: done: 0.039285 sec","job_status":"done","queue_duration_s":1.057091,"scheduling_latency_s":1.057111,"redis_calls":9,"redis_duration_s":0.002113,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000427,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.039285,"completed_at":"2026-10-03T07:06:37.382Z","load_balancing_strategy":"primary","db_duration_s":0.011909,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.13 seconds)2187gitlab # [ 437.616708] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.385Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2fd42181b7178464cf1e9cf6","created_at":"2026-10-03T07:06:36.295Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"beba4f97cd4078e5094338aac7a5e62b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.303Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2fd42181b7178464cf1e9cf6: start","job_status":"start","queue_duration_s":1.081765,"scheduling_latency_s":1.081793}2188gitlab # [ 437.631978] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.405Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2189gitlab # [ 437.636151] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.420Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"2fd42181b7178464cf1e9cf6","created_at":"2026-10-03T07:06:36.295Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"beba4f97cd4078e5094338aac7a5e62b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.303Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-2fd42181b7178464cf1e9cf6: done: 0.034793 sec","job_status":"done","queue_duration_s":1.081765,"scheduling_latency_s":1.081793,"redis_calls":9,"redis_duration_s":0.002649,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000612,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001566,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034793,"completed_at":"2026-10-03T07:06:37.419Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000585,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190gitlab # [ 437.680388] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.421Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6dcfaaed46b16ea2d4f23c45","created_at":"2026-10-03T07:06:36.313Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67b5675ea3f140a2077d6f22fc663b22","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-03T07:06:36.326Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6dcfaaed46b16ea2d4f23c45: start","job_status":"start","queue_duration_s":1.094825,"scheduling_latency_s":1.09485}2191gitlab # [ 437.695152] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.438Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2192gitlab # [ 437.699495] gitaly[934]: time="2026-10-03T07:06:37.495Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C4FHDJBKXN3MQ4HTTVAN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:07.394" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:37.495" grpc.time_ms=0.622 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2193gitlab # [ 437.713242] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 962194gitlab # [ 437.715944] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.447Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"6dcfaaed46b16ea2d4f23c45","created_at":"2026-10-03T07:06:36.313Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67b5675ea3f140a2077d6f22fc663b22","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-03T07:06:36.326Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-6dcfaaed46b16ea2d4f23c45: done: 0.026033 sec","job_status":"done","queue_duration_s":1.094825,"scheduling_latency_s":1.09485,"redis_calls":9,"redis_duration_s":0.0033109999999999997,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000562,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002352,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026033,"completed_at":"2026-10-03T07:06:37.447Z","load_balancing_strategy":"primary","db_duration_s":0.000866,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 437.759182] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.449Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d58298e821c5e967917c43b4","created_at":"2026-10-03T07:06:36.337Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74a21a7b72649b682010fe7cc96450da","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.351Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d58298e821c5e967917c43b4: start","job_status":"start","queue_duration_s":1.097758,"scheduling_latency_s":1.097778}2196gitlab # [ 437.774293] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"d58298e821c5e967917c43b4","created_at":"2026-10-03T07:06:36.337Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74a21a7b72649b682010fe7cc96450da","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.351Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-d58298e821c5e967917c43b4: done: 0.028227 sec","job_status":"done","queue_duration_s":1.097758,"scheduling_latency_s":1.097778,"redis_calls":11,"redis_duration_s":0.002954,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051996,"exclusive_lock_hold_duration_s":0.00299936,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.028227,"completed_at":"2026-10-03T07:06:37.477Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001647,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2197gitlab # [ 437.819188] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.478Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2dfabbb95c9b8dd9e683f88b","created_at":"2026-10-03T07:06:36.374Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9f0dad7681022b793b78077fc3ccfc0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.385Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2dfabbb95c9b8dd9e683f88b: start","job_status":"start","queue_duration_s":1.093543,"scheduling_latency_s":1.093562}2198gitlab # [ 437.834389] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.491Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2199gitlab # [ 437.838181] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"2dfabbb95c9b8dd9e683f88b","created_at":"2026-10-03T07:06:36.374Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9f0dad7681022b793b78077fc3ccfc0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.385Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-2dfabbb95c9b8dd9e683f88b: done: 0.021298 sec","job_status":"done","queue_duration_s":1.093543,"scheduling_latency_s":1.093562,"redis_calls":9,"redis_duration_s":0.001449,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000332,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021298,"completed_at":"2026-10-03T07:06:37.499Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000519,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2200gitlab # [ 437.882693] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.501Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5b7ca63c3c03825ccaaa285f","created_at":"2026-10-03T07:06:36.413Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddf32be9670319f8ad423b09c5e5ec9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-03T07:06:36.425Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5b7ca63c3c03825ccaaa285f: start","job_status":"start","queue_duration_s":1.075948,"scheduling_latency_s":1.075966}2201gitlab # [ 437.896602] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.535Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"5b7ca63c3c03825ccaaa285f","created_at":"2026-10-03T07:06:36.413Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddf32be9670319f8ad423b09c5e5ec9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-03T07:06:36.425Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-5b7ca63c3c03825ccaaa285f: done: 0.034506 sec","job_status":"done","queue_duration_s":1.075948,"scheduling_latency_s":1.075966,"redis_calls":9,"redis_duration_s":0.001513,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000314,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000326,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034506,"completed_at":"2026-10-03T07:06:37.535Z","load_balancing_strategy":"primary","db_duration_s":0.003334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 437.941512] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5ddd887876df842d7688959d","created_at":"2026-10-03T07:06:36.445Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59cbedcc7432c74fb63e0dd1875b8382","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.460Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5ddd887876df842d7688959d: start","job_status":"start","queue_duration_s":1.077267,"scheduling_latency_s":1.077289}2203gitlab # [ 437.956939] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.563Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"5ddd887876df842d7688959d","created_at":"2026-10-03T07:06:36.445Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"59cbedcc7432c74fb63e0dd1875b8382","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.460Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-5ddd887876df842d7688959d: done: 0.025601 sec","job_status":"done","queue_duration_s":1.077267,"scheduling_latency_s":1.077289,"redis_calls":11,"redis_duration_s":0.001743,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000305,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016877,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025601,"completed_at":"2026-10-03T07:06:37.563Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2204gitlab # [ 438.000173] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7484a07cbef238592ca454a8","created_at":"2026-10-03T07:06:36.472Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14e70bfffd18d8bb22540a45de988562","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.484Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7484a07cbef238592ca454a8: start","job_status":"start","queue_duration_s":1.080279,"scheduling_latency_s":1.080298}2205gitlab # [ 438.014558] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7484a07cbef238592ca454a8","created_at":"2026-10-03T07:06:36.472Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"14e70bfffd18d8bb22540a45de988562","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.484Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7484a07cbef238592ca454a8: done: 0.027575 sec","job_status":"done","queue_duration_s":1.080279,"scheduling_latency_s":1.080298,"redis_calls":9,"redis_duration_s":0.0019500000000000001,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00045,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001025,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000475,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027575,"completed_at":"2026-10-03T07:06:37.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003018,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206gitlab # [ 438.057852] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6e499454dbf522a35261398e","created_at":"2026-10-03T07:06:36.498Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0afb9f62374132e5e0cf1a1896e7f11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-03T07:06:36.510Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6e499454dbf522a35261398e: start","job_status":"start","queue_duration_s":1.082993,"scheduling_latency_s":1.083011}2207gitlab # [ 438.071778] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.606Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2208gitlab # [ 438.075391] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6e499454dbf522a35261398e","created_at":"2026-10-03T07:06:36.498Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0afb9f62374132e5e0cf1a1896e7f11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-03T07:06:36.510Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6e499454dbf522a35261398e: done: 0.020399 sec","job_status":"done","queue_duration_s":1.082993,"scheduling_latency_s":1.083011,"redis_calls":9,"redis_duration_s":0.001819,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020399,"completed_at":"2026-10-03T07:06:37.614Z","load_balancing_strategy":"primary","db_duration_s":0.000681,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2209gitlab # [ 438.118800] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"cd135b32179e15d840a3ed78","created_at":"2026-10-03T07:06:36.522Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"daee0d8969e44dad9e0997a2c12255a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.534Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-cd135b32179e15d840a3ed78: start","job_status":"start","queue_duration_s":1.080966,"scheduling_latency_s":1.080983}2210gitlab # [ 438.133841] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.679Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"cd135b32179e15d840a3ed78","created_at":"2026-10-03T07:06:36.522Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"daee0d8969e44dad9e0997a2c12255a5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.534Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-cd135b32179e15d840a3ed78: done: 0.06387 sec","job_status":"done","queue_duration_s":1.080966,"scheduling_latency_s":1.080983,"redis_calls":13,"redis_duration_s":0.003567,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000745,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001842,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052424,"exclusive_lock_hold_duration_s":0.03898022,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.06387,"completed_at":"2026-10-03T07:06:37.679Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2211gitlab # [ 438.180121] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.681Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1f20def554f8a0fa408b09ed","created_at":"2026-10-03T07:06:36.545Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6cef3c6705d592a45bb0c72008ca9f3c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.558Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1f20def554f8a0fa408b09ed: start","job_status":"start","queue_duration_s":1.122147,"scheduling_latency_s":1.122164}2212gitlab # [ 438.195223] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1f20def554f8a0fa408b09ed","created_at":"2026-10-03T07:06:36.545Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6cef3c6705d592a45bb0c72008ca9f3c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.558Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1f20def554f8a0fa408b09ed: done: 0.020554 sec","job_status":"done","queue_duration_s":1.122147,"scheduling_latency_s":1.122164,"redis_calls":9,"redis_duration_s":0.0018989999999999999,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000395,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020554,"completed_at":"2026-10-03T07:06:37.701Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000472,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2213gitlab # [ 438.239374] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.703Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"203846cea32333693a7590e4","created_at":"2026-10-03T07:06:36.574Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e52037a36f27769a0c5aa7424963ea9f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.585Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-203846cea32333693a7590e4: start","job_status":"start","queue_duration_s":1.117084,"scheduling_latency_s":1.117101}2214gitlab # [ 438.254798] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"203846cea32333693a7590e4","created_at":"2026-10-03T07:06:36.574Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e52037a36f27769a0c5aa7424963ea9f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.585Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-203846cea32333693a7590e4: done: 0.046134 sec","job_status":"done","queue_duration_s":1.117084,"scheduling_latency_s":1.117101,"redis_calls":11,"redis_duration_s":0.002814,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001319,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00072772,"exclusive_lock_hold_duration_s":0.01382362,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.046134,"completed_at":"2026-10-03T07:06:37.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013725,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2215gitlab # [ 438.301070] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:37.750Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"55b68fd3a427b807735d761c","created_at":"2026-10-03T07:06:36.598Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f45582ef30564206bcfde0bb7acf121f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.610Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-55b68fd3a427b807735d761c: start","job_status":"start","queue_duration_s":1.139924,"scheduling_latency_s":1.139941}2216gitlab # [ 438.315439] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"55b68fd3a427b807735d761c","created_at":"2026-10-03T07:06:36.598Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f45582ef30564206bcfde0bb7acf121f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.610Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-55b68fd3a427b807735d761c: done: 0.281629 sec","job_status":"done","queue_duration_s":1.139924,"scheduling_latency_s":1.139941,"redis_calls":13,"redis_duration_s":0.00344,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000882,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001518,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.232312,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063718,"exclusive_lock_hold_duration_s":0.25128038,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.281629,"completed_at":"2026-10-03T07:06:38.032Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056959,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2217gitlab # [ 438.362385] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"de6ba85198a5a07a5a76317e","created_at":"2026-10-03T07:06:36.644Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc448626773046887b7284e8e409683a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.659Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-de6ba85198a5a07a5a76317e: start","job_status":"start","queue_duration_s":1.375007,"scheduling_latency_s":1.375029}2218gitlab # [ 438.377473] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.076Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"de6ba85198a5a07a5a76317e","created_at":"2026-10-03T07:06:36.644Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc448626773046887b7284e8e409683a","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.659Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-de6ba85198a5a07a5a76317e: done: 0.041746 sec","job_status":"done","queue_duration_s":1.375007,"scheduling_latency_s":1.375029,"redis_calls":12,"redis_duration_s":0.003759,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000539,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001917,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001303,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01144846,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.041746,"completed_at":"2026-10-03T07:06:38.076Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003339,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2219gitlab # [ 438.423344] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.077Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"4e5e38379d7fee39c09dda95","created_at":"2026-10-03T07:06:36.670Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50541033ff47eb9c362b5226770d5ea5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.683Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-4e5e38379d7fee39c09dda95: start","job_status":"start","queue_duration_s":1.3942,"scheduling_latency_s":1.394222}2220gitlab # [ 438.438457] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"4e5e38379d7fee39c09dda95","created_at":"2026-10-03T07:06:36.670Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50541033ff47eb9c362b5226770d5ea5","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.683Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-4e5e38379d7fee39c09dda95: done: 0.02591 sec","job_status":"done","queue_duration_s":1.3942,"scheduling_latency_s":1.394222,"redis_calls":12,"redis_duration_s":0.002466,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031864,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.02591,"completed_at":"2026-10-03T07:06:38.103Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2221gitlab # [ 438.484811] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.105Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"95f209bce1b61b7513281ddf","created_at":"2026-10-03T07:06:36.696Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c055fcfc16115453016b5417525a919","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.708Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-95f209bce1b61b7513281ddf: start","job_status":"start","queue_duration_s":1.39633,"scheduling_latency_s":1.39635}2222gitlab # [ 438.498290] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"95f209bce1b61b7513281ddf","created_at":"2026-10-03T07:06:36.696Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6c055fcfc16115453016b5417525a919","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.708Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-95f209bce1b61b7513281ddf: done: 0.020241 sec","job_status":"done","queue_duration_s":1.39633,"scheduling_latency_s":1.39635,"redis_calls":11,"redis_duration_s":0.002138,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000429,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000382,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020241,"completed_at":"2026-10-03T07:06:38.125Z","load_balancing_strategy":"primary","db_duration_s":0.000409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2223gitlab # [ 438.542684] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.127Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9e1c8c65e12a5f865e652d62","created_at":"2026-10-03T07:06:36.726Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"103a7f92642726e297d50f48d51c3405","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.738Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9e1c8c65e12a5f865e652d62: start","job_status":"start","queue_duration_s":1.388605,"scheduling_latency_s":1.388623}2224gitlab # [ 438.557859] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.138Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2225gitlab # [ 438.561401] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9e1c8c65e12a5f865e652d62","created_at":"2026-10-03T07:06:36.726Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"103a7f92642726e297d50f48d51c3405","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.738Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9e1c8c65e12a5f865e652d62: done: 0.019156 sec","job_status":"done","queue_duration_s":1.388605,"scheduling_latency_s":1.388623,"redis_calls":9,"redis_duration_s":0.00203,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000367,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019156,"completed_at":"2026-10-03T07:06:38.146Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2226gitlab # [ 438.605054] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"74810ef18da56dc64953beca","created_at":"2026-10-03T07:06:36.903Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.916Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-74810ef18da56dc64953beca: start","job_status":"start","queue_duration_s":1.231462,"scheduling_latency_s":1.23148}2227gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2228gitlab # [ 438.618902] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"74810ef18da56dc64953beca","created_at":"2026-10-03T07:06:36.903Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.916Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-74810ef18da56dc64953beca: done: 0.108868 sec","job_status":"done","queue_duration_s":1.231462,"scheduling_latency_s":1.23148,"redis_calls":70,"redis_duration_s":0.015671,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001918,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000858,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.012343,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.108868,"completed_at":"2026-10-03T07:06:38.256Z","load_balancing_strategy":"primary","db_duration_s":0.001713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229gitlab # [ 438.661839] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.257Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d1ae3be28f21804660279a84","created_at":"2026-10-03T07:06:36.929Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"243a581e0cb6ba37315e935d3d4d30ae","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.939Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d1ae3be28f21804660279a84: start","job_status":"start","queue_duration_s":1.318336,"scheduling_latency_s":1.318353}2230gitlab # [ 438.676097] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.286Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"d1ae3be28f21804660279a84","created_at":"2026-10-03T07:06:36.929Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"243a581e0cb6ba37315e935d3d4d30ae","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.939Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-d1ae3be28f21804660279a84: done: 0.028101 sec","job_status":"done","queue_duration_s":1.318336,"scheduling_latency_s":1.318353,"redis_calls":11,"redis_duration_s":0.002345,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001036,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053342,"exclusive_lock_hold_duration_s":0.00685538,"duration_s":0.028101,"completed_at":"2026-10-03T07:06:38.285Z","load_balancing_strategy":"primary","db_duration_s":0.00349,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab # [ 438.718934] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.287Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"31a3c6bff5a8b9b728fe8229","created_at":"2026-10-03T07:06:36.950Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a7690b696609bfb5d734efb880b2582","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.962Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-31a3c6bff5a8b9b728fe8229: start","job_status":"start","queue_duration_s":1.324939,"scheduling_latency_s":1.324957}2232gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.14 seconds)2233gitlab # [ 438.734005] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"31a3c6bff5a8b9b728fe8229","created_at":"2026-10-03T07:06:36.950Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a7690b696609bfb5d734efb880b2582","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.962Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-31a3c6bff5a8b9b728fe8229: done: 0.029482 sec","job_status":"done","queue_duration_s":1.324939,"scheduling_latency_s":1.324957,"redis_calls":9,"redis_duration_s":0.001782,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000388,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000996,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029482,"completed_at":"2026-10-03T07:06:38.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2234gitlab # [ 438.778511] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a6053fe8fb46f5363aa4c33e","created_at":"2026-10-03T07:06:36.973Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cbb502c41830b35bf3bd87da3c6aa6c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.984Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a6053fe8fb46f5363aa4c33e: start","job_status":"start","queue_duration_s":1.33388,"scheduling_latency_s":1.333897}2235gitlab # [ 438.792834] gitaly[934]: time="2026-10-03T07:06:38.631Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C5KH4XZA5NT9H831FFC1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:08.529" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:38.630" grpc.time_ms=0.765 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2236gitlab # [ 438.806439] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:38 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 852237gitlab # [ 438.809230] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"a6053fe8fb46f5363aa4c33e","created_at":"2026-10-03T07:06:36.973Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5cbb502c41830b35bf3bd87da3c6aa6c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:36.984Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-a6053fe8fb46f5363aa4c33e: done: 0.037539 sec","job_status":"done","queue_duration_s":1.33388,"scheduling_latency_s":1.333897,"redis_calls":18,"redis_duration_s":0.0034809999999999997,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000904,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000259,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001705,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037539,"completed_at":"2026-10-03T07:06:38.356Z","load_balancing_strategy":"primary","db_duration_s":0.002246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2238gitlab # [ 438.852932] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.358Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"164fa85f1efe5ad3f7f87b63","created_at":"2026-10-03T07:06:37.002Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbb05150c715dd49e4f794d5a02c8c63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:37.013Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-164fa85f1efe5ad3f7f87b63: start","job_status":"start","queue_duration_s":1.344105,"scheduling_latency_s":1.344123}2239gitlab # [ 438.867934] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.373Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2240gitlab # [ 438.871458] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.387Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"164fa85f1efe5ad3f7f87b63","created_at":"2026-10-03T07:06:37.002Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-fd11f959016045d7","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbb05150c715dd49e4f794d5a02c8c63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-03T07:06:37.013Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-164fa85f1efe5ad3f7f87b63: done: 0.029394 sec","job_status":"done","queue_duration_s":1.344105,"scheduling_latency_s":1.344123,"redis_calls":9,"redis_duration_s":0.0022849999999999997,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000602,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029394,"completed_at":"2026-10-03T07:06:38.387Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 438.915474] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca8175755c69778ea7633f45","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca8175755c69778ea7633f45: start","job_status":"start","queue_duration_s":0.142286,"scheduling_latency_s":0.142311}2242gitlab # [ 438.928305] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca8175755c69778ea7633f45","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca8175755c69778ea7633f45: done: 0.035627 sec","job_status":"done","queue_duration_s":0.142286,"scheduling_latency_s":0.142311,"redis_calls":15,"redis_duration_s":0.003294,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001845,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022284,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035627,"completed_at":"2026-10-03T07:06:38.425Z","load_balancing_strategy":"primary","db_duration_s":0.000504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2243gitlab # [ 438.970470] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.427Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d9ec05cc414102546ab91ab","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d9ec05cc414102546ab91ab: start","job_status":"start","queue_duration_s":0.179615,"scheduling_latency_s":0.179637}2244gitlab # [ 438.984371] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d9ec05cc414102546ab91ab","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d9ec05cc414102546ab91ab: done: 0.020928 sec","job_status":"done","queue_duration_s":0.179615,"scheduling_latency_s":0.179637,"redis_calls":13,"redis_duration_s":0.002496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001669,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020928,"completed_at":"2026-10-03T07:06:38.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 439.024884] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e2641707b005fb83b285f69","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e2641707b005fb83b285f69: start","job_status":"start","queue_duration_s":0.202504,"scheduling_latency_s":0.202524}2246gitlab # [ 439.037735] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e2641707b005fb83b285f69","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e2641707b005fb83b285f69: done: 0.018692 sec","job_status":"done","queue_duration_s":0.202504,"scheduling_latency_s":0.202524,"redis_calls":13,"redis_duration_s":0.002307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001529,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018692,"completed_at":"2026-10-03T07:06:38.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2247gitlab # [ 439.078437] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7715d8774232b74ce9806869","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7715d8774232b74ce9806869: start","job_status":"start","queue_duration_s":0.223049,"scheduling_latency_s":0.223067}2248gitlab # [ 439.091259] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7715d8774232b74ce9806869","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7715d8774232b74ce9806869: done: 0.017286 sec","job_status":"done","queue_duration_s":0.223049,"scheduling_latency_s":0.223067,"redis_calls":13,"redis_duration_s":0.00224,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017286,"completed_at":"2026-10-03T07:06:38.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2249gitlab # [ 439.131939] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a208b5aec8590ffd5aa8b4d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a208b5aec8590ffd5aa8b4d: start","job_status":"start","queue_duration_s":0.242251,"scheduling_latency_s":0.24227}2250gitlab # [ 439.145147] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a208b5aec8590ffd5aa8b4d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a208b5aec8590ffd5aa8b4d: done: 0.016241 sec","job_status":"done","queue_duration_s":0.242251,"scheduling_latency_s":0.24227,"redis_calls":13,"redis_duration_s":0.002112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001392,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016241,"completed_at":"2026-10-03T07:06:38.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2251gitlab # [ 439.186330] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcaaf00c98dc04263b6cc9f8","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcaaf00c98dc04263b6cc9f8: start","job_status":"start","queue_duration_s":0.260022,"scheduling_latency_s":0.260039}2252gitlab # [ 439.199321] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcaaf00c98dc04263b6cc9f8","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcaaf00c98dc04263b6cc9f8: done: 0.015997 sec","job_status":"done","queue_duration_s":0.260022,"scheduling_latency_s":0.260039,"redis_calls":13,"redis_duration_s":0.002044,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015997,"completed_at":"2026-10-03T07:06:38.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2253gitlab # [ 439.239877] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c974b93427d740f76f377038","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c974b93427d740f76f377038: start","job_status":"start","queue_duration_s":0.277737,"scheduling_latency_s":0.277756}2254gitlab # [ 439.252949] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c974b93427d740f76f377038","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c974b93427d740f76f377038: done: 0.01618 sec","job_status":"done","queue_duration_s":0.277737,"scheduling_latency_s":0.277756,"redis_calls":13,"redis_duration_s":0.002344,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001603,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01618,"completed_at":"2026-10-03T07:06:38.541Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2255gitlab # [ 439.293467] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71f56eb103855659520532dd","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71f56eb103855659520532dd: start","job_status":"start","queue_duration_s":0.295126,"scheduling_latency_s":0.295143}2256gitlab # [ 439.306482] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71f56eb103855659520532dd","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71f56eb103855659520532dd: done: 0.019865 sec","job_status":"done","queue_duration_s":0.295126,"scheduling_latency_s":0.295143,"redis_calls":13,"redis_duration_s":0.00434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002278,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012865,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019865,"completed_at":"2026-10-03T07:06:38.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab # [ 439.347863] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bca893fe5bc553bc72b3d5d2","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bca893fe5bc553bc72b3d5d2: start","job_status":"start","queue_duration_s":0.316273,"scheduling_latency_s":0.316289}2258gitlab # [ 439.360708] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bca893fe5bc553bc72b3d5d2","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bca893fe5bc553bc72b3d5d2: done: 0.022573 sec","job_status":"done","queue_duration_s":0.316273,"scheduling_latency_s":0.316289,"redis_calls":13,"redis_duration_s":0.0050550000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002362,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022573,"completed_at":"2026-10-03T07:06:38.586Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259gitlab # [ 439.402037] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"646cc11158dc88430b3ca6ef","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-646cc11158dc88430b3ca6ef: start","job_status":"start","queue_duration_s":0.340598,"scheduling_latency_s":0.340622}2260gitlab # [ 439.415146] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"646cc11158dc88430b3ca6ef","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-646cc11158dc88430b3ca6ef: done: 0.023721 sec","job_status":"done","queue_duration_s":0.340598,"scheduling_latency_s":0.340622,"redis_calls":13,"redis_duration_s":0.003033,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002112,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023721,"completed_at":"2026-10-03T07:06:38.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2261gitlab # [ 439.456256] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22a5b4991d9844e4bd55d9d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22a5b4991d9844e4bd55d9d: start","job_status":"start","queue_duration_s":0.365938,"scheduling_latency_s":0.36596}2262gitlab # [ 439.469458] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d22a5b4991d9844e4bd55d9d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d22a5b4991d9844e4bd55d9d: done: 0.017658 sec","job_status":"done","queue_duration_s":0.365938,"scheduling_latency_s":0.36596,"redis_calls":13,"redis_duration_s":0.0020889999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000748,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001341,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017658,"completed_at":"2026-10-03T07:06:38.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263gitlab # [ 439.511286] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b13764dcc278b8fd3eeea628","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b13764dcc278b8fd3eeea628: start","job_status":"start","queue_duration_s":0.384877,"scheduling_latency_s":0.384893}2264gitlab # [ 439.524710] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b13764dcc278b8fd3eeea628","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b13764dcc278b8fd3eeea628: done: 0.021421 sec","job_status":"done","queue_duration_s":0.384877,"scheduling_latency_s":0.384893,"redis_calls":13,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001596,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021421,"completed_at":"2026-10-03T07:06:38.654Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2265gitlab # [ 439.565431] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ad0798b01447ccad7ca892e","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ad0798b01447ccad7ca892e: start","job_status":"start","queue_duration_s":0.408214,"scheduling_latency_s":0.408232}2266gitlab # [ 439.578334] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ad0798b01447ccad7ca892e","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ad0798b01447ccad7ca892e: done: 0.017932 sec","job_status":"done","queue_duration_s":0.408214,"scheduling_latency_s":0.408232,"redis_calls":13,"redis_duration_s":0.001876,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017932,"completed_at":"2026-10-03T07:06:38.674Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2267gitlab # [ 439.620339] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6390745205670cab35a3819f","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6390745205670cab35a3819f: start","job_status":"start","queue_duration_s":0.428048,"scheduling_latency_s":0.428068}2268gitlab # [ 439.633452] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6390745205670cab35a3819f","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6390745205670cab35a3819f: done: 0.01798 sec","job_status":"done","queue_duration_s":0.428048,"scheduling_latency_s":0.428068,"redis_calls":13,"redis_duration_s":0.002317,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000784,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01798,"completed_at":"2026-10-03T07:06:38.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2269gitlab # [ 439.673972] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8fc4941dec3b15e9425b478","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8fc4941dec3b15e9425b478: start","job_status":"start","queue_duration_s":0.447945,"scheduling_latency_s":0.447965}2270gitlab # [ 439.686801] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8fc4941dec3b15e9425b478","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8fc4941dec3b15e9425b478: done: 0.017894 sec","job_status":"done","queue_duration_s":0.447945,"scheduling_latency_s":0.447965,"redis_calls":13,"redis_duration_s":0.00235,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001514,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012224,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017894,"completed_at":"2026-10-03T07:06:38.713Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2271gitlab # [ 439.727260] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb52ffd84a42492466588154","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb52ffd84a42492466588154: start","job_status":"start","queue_duration_s":0.46722,"scheduling_latency_s":0.467239}2272gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2273gitlab # [ 439.740060] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb52ffd84a42492466588154","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb52ffd84a42492466588154: done: 0.015382 sec","job_status":"done","queue_duration_s":0.46722,"scheduling_latency_s":0.467239,"redis_calls":13,"redis_duration_s":0.002102,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000746,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001356,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015382,"completed_at":"2026-10-03T07:06:38.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2274gitlab # [ 439.779538] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34a69f75fa8f3de63e8b074d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34a69f75fa8f3de63e8b074d: start","job_status":"start","queue_duration_s":0.483975,"scheduling_latency_s":0.483995}2275gitlab # [ 439.791839] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34a69f75fa8f3de63e8b074d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34a69f75fa8f3de63e8b074d: done: 0.015075 sec","job_status":"done","queue_duration_s":0.483975,"scheduling_latency_s":0.483995,"redis_calls":13,"redis_duration_s":0.00213,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015075,"completed_at":"2026-10-03T07:06:38.746Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 439.833207] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55be3c28caf450bf448e8b7b","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55be3c28caf450bf448e8b7b: start","job_status":"start","queue_duration_s":0.500267,"scheduling_latency_s":0.500283}2277gitlab # [ 439.845948] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.763Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55be3c28caf450bf448e8b7b","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55be3c28caf450bf448e8b7b: done: 0.014916 sec","job_status":"done","queue_duration_s":0.500267,"scheduling_latency_s":0.500283,"redis_calls":13,"redis_duration_s":0.002057,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001307,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014916,"completed_at":"2026-10-03T07:06:38.763Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2278gitlab # [ 439.886320] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d135fe9c51eb62d3a11dcf16","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d135fe9c51eb62d3a11dcf16: start","job_status":"start","queue_duration_s":0.516478,"scheduling_latency_s":0.516494}2279gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.16 seconds)2280gitlab # [ 439.899088] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d135fe9c51eb62d3a11dcf16","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d135fe9c51eb62d3a11dcf16: done: 0.014831 sec","job_status":"done","queue_duration_s":0.516478,"scheduling_latency_s":0.516494,"redis_calls":13,"redis_duration_s":0.0018679999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001158,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014831,"completed_at":"2026-10-03T07:06:38.779Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281gitlab # [ 439.940296] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef8bb3b599233684b650438f","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef8bb3b599233684b650438f: start","job_status":"start","queue_duration_s":0.532427,"scheduling_latency_s":0.532444}2282gitlab # [ 439.953465] gitaly[934]: time="2026-10-03T07:06:39.788Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C6Q748TM2SWASMHZJS2X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:09.786" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:39.787" grpc.time_ms=0.907 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2283gitlab # [ 439.967055] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1042284gitlab # [ 439.970195] nix-daemon[3765]: accepted connection from pid 4855, user nixbot2285gitlab # [ 439.971181] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef8bb3b599233684b650438f","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef8bb3b599233684b650438f: done: 0.014635 sec","job_status":"done","queue_duration_s":0.532427,"scheduling_latency_s":0.532444,"redis_calls":13,"redis_duration_s":0.001907,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014635,"completed_at":"2026-10-03T07:06:38.794Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2286gitlab # [ 440.012146] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc49bed27fb176f797a94031","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc49bed27fb176f797a94031: start","job_status":"start","queue_duration_s":0.548279,"scheduling_latency_s":0.548296}2287gitlab # [ 440.024676] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc49bed27fb176f797a94031","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc49bed27fb176f797a94031: done: 0.014515 sec","job_status":"done","queue_duration_s":0.548279,"scheduling_latency_s":0.548296,"redis_calls":13,"redis_duration_s":0.001935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001225,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014515,"completed_at":"2026-10-03T07:06:38.810Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2288gitlab # [ 440.066048] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a26c3c9bd675ee810836a9c8","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a26c3c9bd675ee810836a9c8: start","job_status":"start","queue_duration_s":0.563987,"scheduling_latency_s":0.564004}2289gitlab # [ 440.079062] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a26c3c9bd675ee810836a9c8","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a26c3c9bd675ee810836a9c8: done: 0.014739 sec","job_status":"done","queue_duration_s":0.563987,"scheduling_latency_s":0.564004,"redis_calls":13,"redis_duration_s":0.001976,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001291,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014739,"completed_at":"2026-10-03T07:06:38.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2290gitlab # [ 440.119857] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"465b877eac6ee50fa6c9290a","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-465b877eac6ee50fa6c9290a: start","job_status":"start","queue_duration_s":0.579925,"scheduling_latency_s":0.579942}2291gitlab # [ 440.132689] gitaly[934]: time="2026-10-03T07:06:39.925Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:09.024" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:39.924" grpc.time_ms=0.742 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2292gitlab # [ 440.145757] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"465b877eac6ee50fa6c9290a","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-465b877eac6ee50fa6c9290a: done: 0.014697 sec","job_status":"done","queue_duration_s":0.579925,"scheduling_latency_s":0.579942,"redis_calls":13,"redis_duration_s":0.0019779999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001285,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014697,"completed_at":"2026-10-03T07:06:38.842Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2293gitlab # [ 440.186383] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"494462429ac5459450283e6d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-494462429ac5459450283e6d: start","job_status":"start","queue_duration_s":0.595818,"scheduling_latency_s":0.595835}2294gitlab # [ 440.199905] nixbot[3609]: {"ts": "2026-10-03T07:06:40.041Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 \"HTTP/1.1 201 Created\""}2295gitlab # [ 440.202991] gitaly[934]: time="2026-10-03T07:06:39.973Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:49.172" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:39.962" grpc.time_ms=10.881 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2296gitlab # [ 440.217067] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 1802297gitlab # [ 440.219634] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"494462429ac5459450283e6d","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-494462429ac5459450283e6d: done: 0.014879 sec","job_status":"done","queue_duration_s":0.595818,"scheduling_latency_s":0.595835,"redis_calls":13,"redis_duration_s":0.002179,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001429,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014879,"completed_at":"2026-10-03T07:06:38.858Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2298gitlab # [ 440.261096] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc9aa318157a8d64858256eb","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc9aa318157a8d64858256eb: start","job_status":"start","queue_duration_s":0.611951,"scheduling_latency_s":0.611968}2299gitlab # [ 440.273967] gitaly[934]: time="2026-10-03T07:06:40.108Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C71EA5G4931FYF0137WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:10.006" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:40.107" grpc.time_ms=0.74 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2300gitlab # [ 440.287427] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc9aa318157a8d64858256eb","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc9aa318157a8d64858256eb: done: 0.014825 sec","job_status":"done","queue_duration_s":0.611951,"scheduling_latency_s":0.611968,"redis_calls":13,"redis_duration_s":0.002035,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009866,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014825,"completed_at":"2026-10-03T07:06:38.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2301gitlab # [ 440.328294] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35869ef2f4ec61e217f7cf9e","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35869ef2f4ec61e217f7cf9e: start","job_status":"start","queue_duration_s":0.62806,"scheduling_latency_s":0.628077}2302gitlab # [ 440.341134] gitaly[934]: time="2026-10-03T07:06:40.152Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M409C71EA5G4931FYF0137WA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:50.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.142" grpc.time_ms=10.158 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2303gitlab # [ 440.355467] gitaly[934]: time="2026-10-03T07:06:40.167Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:40.166" grpc.time_ms=0.638 pid=934 span.kind=server system=grpc2304gitlab # [ 440.365777] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1552305gitlab # [ 440.368870] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35869ef2f4ec61e217f7cf9e","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35869ef2f4ec61e217f7cf9e: done: 0.015119 sec","job_status":"done","queue_duration_s":0.62806,"scheduling_latency_s":0.628077,"redis_calls":13,"redis_duration_s":0.002328,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001565,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015119,"completed_at":"2026-10-03T07:06:38.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2306gitlab # [ 440.410162] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"677f3fe3081f207a3cd0e2e7","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-677f3fe3081f207a3cd0e2e7: start","job_status":"start","queue_duration_s":0.644426,"scheduling_latency_s":0.644443}2307gitlab # [ 440.423748] nixbot[3609]: {"ts": "2026-10-03T07:06:40.203Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 \"HTTP/1.1 201 Created\""}2308gitlab # [ 440.426968] gitaly[934]: time="2026-10-03T07:06:40.199Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:40.198" grpc.time_ms=0.519 pid=934 span.kind=server system=grpc2309gitlab # [ 440.437413] gitaly[934]: time="2026-10-03T07:06:40.233Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:40.233" grpc.time_ms=0.573 pid=934 span.kind=server system=grpc2310gitlab # [ 440.447887] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"677f3fe3081f207a3cd0e2e7","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-677f3fe3081f207a3cd0e2e7: done: 0.014682 sec","job_status":"done","queue_duration_s":0.644426,"scheduling_latency_s":0.644443,"redis_calls":13,"redis_duration_s":0.001986,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00071,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001276,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014682,"completed_at":"2026-10-03T07:06:38.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2311gitlab # [ 440.490315] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d240da1c150c60b23ef5c8ab","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d240da1c150c60b23ef5c8ab: start","job_status":"start","queue_duration_s":0.660285,"scheduling_latency_s":0.660301}2312gitlab # [ 440.503499] gitaly[934]: time="2026-10-03T07:06:40.239Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:07:10.133" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.134" operations="[repack_refs]" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2313gitlab # [ 440.513716] gitaly[934]: time="2026-10-03T07:06:40.242Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:07:10.133" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.134" pid=934 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-03 07:04:43.18333992 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2314gitlab # [ 440.526373] nixbot[3609]: {"ts": "2026-10-03T07:06:40.362Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 \"HTTP/1.1 201 Created\""}2315gitlab # [ 440.529328] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d240da1c150c60b23ef5c8ab","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d240da1c150c60b23ef5c8ab: done: 0.014853 sec","job_status":"done","queue_duration_s":0.660285,"scheduling_latency_s":0.660301,"redis_calls":13,"redis_duration_s":0.002003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001268,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014853,"completed_at":"2026-10-03T07:06:38.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2316gitlab # [ 440.570073] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90d1eff0d84df5c1ccfbc921","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90d1eff0d84df5c1ccfbc921: start","job_status":"start","queue_duration_s":0.676457,"scheduling_latency_s":0.676474}2317gitlab # [ 440.583183] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:40 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/e874471c6bba3c76d684eaabbff376e5652e08d0 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1452318gitlab # [ 440.586018] gitaly[934]: time="2026-10-03T07:06:40.245Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S diskcache=a3c5c3aa-3387-452c-9823-5b8130896006 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:07:10.133" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.134" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2319gitlab # [ 440.596041] gitaly[934]: time="2026-10-03T07:06:40.245Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=99 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7098 command.oublock=0 command.real_time_ms=103 command.spawn_token_fork_ms=6 command.system_time_ms=60 command.user_time_ms=39 component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:07:10.133" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.134" grpc.time_ms=111.592 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2320gitlab # [ 440.609707] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90d1eff0d84df5c1ccfbc921","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90d1eff0d84df5c1ccfbc921: done: 0.015084 sec","job_status":"done","queue_duration_s":0.676457,"scheduling_latency_s":0.676474,"redis_calls":13,"redis_duration_s":0.002139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001437,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015084,"completed_at":"2026-10-03T07:06:38.939Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321gitlab # [ 440.650755] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"418f5fec6cbfeba8d410ecd9","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-418f5fec6cbfeba8d410ecd9: start","job_status":"start","queue_duration_s":0.69283,"scheduling_latency_s":0.692847}2322gitlab # [ 440.664998] gitaly[934]: time="2026-10-03T07:06:40.266Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:07:10.133" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.134" pid=934 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2323gitlab # [ 440.679177] gitaly[934]: time="2026-10-03T07:06:40.267Z" level=info msg="optimized repository" clean_stale_data_time=24.58718ms clean_work_trees_time="125.42µs" commit_graph_time=660ns correlation_id=01M409C6VKNTV17067DJAJE55S gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.64µs" pid=934 prune_time=160ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="1.46µs" storage_name=default system=housekeeping total_time=24.744ms2324gitlab # [ 440.685707] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"418f5fec6cbfeba8d410ecd9","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-418f5fec6cbfeba8d410ecd9: done: 0.015036 sec","job_status":"done","queue_duration_s":0.69283,"scheduling_latency_s":0.692847,"redis_calls":13,"redis_duration_s":0.0021019999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001357,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015036,"completed_at":"2026-10-03T07:06:38.955Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2325gitlab # [ 440.727480] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf22848dd070fba2d41f1232","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf22848dd070fba2d41f1232: start","job_status":"start","queue_duration_s":0.709034,"scheduling_latency_s":0.709052}2326gitlab # [ 440.740621] gitaly[934]: time="2026-10-03T07:06:40.267Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-03T07:07:10.133" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.134" pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2327gitlab # [ 440.750457] gitaly[934]: time="2026-10-03T07:06:40.268Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C76QFJGJ0GNAY3B1XJJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:10.166" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:40.267" grpc.time_ms=0.577 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2328gitlab # [ 440.763933] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf22848dd070fba2d41f1232","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf22848dd070fba2d41f1232: done: 0.014498 sec","job_status":"done","queue_duration_s":0.709034,"scheduling_latency_s":0.709052,"redis_calls":13,"redis_duration_s":0.00197,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000765,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014498,"completed_at":"2026-10-03T07:06:38.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2329gitlab # [ 440.805123] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af5ad89ebfe74a2cf920ca63","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af5ad89ebfe74a2cf920ca63: start","job_status":"start","queue_duration_s":0.724744,"scheduling_latency_s":0.724761}2330gitlab # [ 440.817890] gitaly[934]: time="2026-10-03T07:06:40.314Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409C76QFJGJ0GNAY3B1XJJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-03T07:06:50.163" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-03T07:06:40.303" grpc.time_ms=10.556 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2331gitlab # [ 440.832089] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af5ad89ebfe74a2cf920ca63","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af5ad89ebfe74a2cf920ca63: done: 0.014636 sec","job_status":"done","queue_duration_s":0.724744,"scheduling_latency_s":0.724761,"redis_calls":13,"redis_duration_s":0.0019259999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000726,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014636,"completed_at":"2026-10-03T07:06:38.987Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2332gitlab # [ 440.873703] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:38.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10523750f1ff0d1ef50f560e","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10523750f1ff0d1ef50f560e: start","job_status":"start","queue_duration_s":0.740608,"scheduling_latency_s":0.740626}2333gitlab # [ 440.886735] gitaly[934]: time="2026-10-03T07:06:40.325Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-03T07:07:10.323" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:40.324" grpc.time_ms=0.783 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2334gitlab # [ 440.900037] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:40 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 802335gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses2336gitlab # [ 440.902193] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"10523750f1ff0d1ef50f560e","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-10523750f1ff0d1ef50f560e: done: 0.01489 sec","job_status":"done","queue_duration_s":0.740608,"scheduling_latency_s":0.740626,"redis_calls":13,"redis_duration_s":0.002029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001274,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01489,"completed_at":"2026-10-03T07:06:39.003Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2337gitlab # [ 440.942232] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"157a461245b0759f81091ef0","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-157a461245b0759f81091ef0: start","job_status":"start","queue_duration_s":0.756686,"scheduling_latency_s":0.756703}2338gitlab # [ 440.954822] gitaly[934]: time="2026-10-03T07:06:40.358Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-03T07:07:10.357" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-03T07:06:40.357" grpc.time_ms=0.673 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2339gitlab # [ 440.971890] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"157a461245b0759f81091ef0","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-157a461245b0759f81091ef0: done: 0.014861 sec","job_status":"done","queue_duration_s":0.756686,"scheduling_latency_s":0.756703,"redis_calls":13,"redis_duration_s":0.002121,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000808,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001313,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014861,"completed_at":"2026-10-03T07:06:39.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2340gitlab # [ 441.013241] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"408377d34f1c2b7dc11de143","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-408377d34f1c2b7dc11de143: start","job_status":"start","queue_duration_s":0.772841,"scheduling_latency_s":0.772859}2341gitlab # [ 441.026300] gitaly[934]: time="2026-10-03T07:06:40.762Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M409C7NAX3V45FHRP6CS16QW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-03T07:06:40.749" grpc.time_ms=12.587 pid=934 remote_ip="2001:db8:1::1" span.kind=server system=grpc2342gitlab # [ 441.040221] gitaly[934]: time="2026-10-03T07:06:40.919Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M409C6VKNTV17067DJAJE55S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-03T07:06:40.914" grpc.time_ms=1.303 pid=934 span.kind=server system=grpc2343gitlab # [ 441.050221] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"408377d34f1c2b7dc11de143","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-408377d34f1c2b7dc11de143: done: 0.015104 sec","job_status":"done","queue_duration_s":0.772841,"scheduling_latency_s":0.772859,"redis_calls":13,"redis_duration_s":0.002136,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000721,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015104,"completed_at":"2026-10-03T07:06:39.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2344gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/e874471c6bba3c76d684eaabbff376e5652e08d0/statuses, in 0.17 seconds)2345(finished: subtest: eval and build post commit statuses, in 10.52 seconds)2346(finished: run the VM test script, in 508.17 seconds)2347gitlab # [ 441.092190] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b52c608c93cfee13de061cd4","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b52c608c93cfee13de061cd4: start","job_status":"start","queue_duration_s":0.789229,"scheduling_latency_s":0.789246}2348gitlab # [ 441.108720] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b52c608c93cfee13de061cd4","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b52c608c93cfee13de061cd4: done: 0.015134 sec","job_status":"done","queue_duration_s":0.789229,"scheduling_latency_s":0.789246,"redis_calls":13,"redis_duration_s":0.002474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001412,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015134,"completed_at":"2026-10-03T07:06:39.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2349gitlab # [ 441.155693] sidekiq-cluster[3482]: {"severity":"INFO","time":"2026-10-03T07:06:39.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e7415318e29b04c6a7818ad","created_at":"2026-10-03T07:06:38.167Z","trace_propagation_headers":{"sentry-trace":"9afe25b425764186bd93808ba36576c7-b2d3af984b7643dc","baggage":"sentry-trace_id=9afe25b425764186bd93808ba36576c7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f8cd42919946a03c10582ae2ab7a2756","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-03T07:06:38.247Z","job_size_bytes":2,"pid":3482,"sidekiq_tid":"m8s2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e7415318e29b04c6a7818ad: start","job_status":"start","queue_duration_s":0.80562,"scheduling_latency_s":0.805638}2350test script finished in 508.25s2351gitlab # [ 441.169963] gitlab-workhorse[820]: time="2026-10-03T07:06:40Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672352cleanup2353kill QemuMachine (pid 46)2354gitlab # [ 441.172104] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/10/03:07:06:40 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "gitqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2355(finished: cleanup, in 1.56 seconds)2356additionally exposed symbols:2357 gitlab,2358 vlan1,2359 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2360[]2361[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-03T07:04:55.891Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2362[]2363[]2364[]2365[]2366[]2367[]2368[]2369[]2370[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-03T07:06:29.717623Z', 'started_at': '2026-10-03T07:06:30.088688Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2371[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2372[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2373[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2374[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2375[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2376[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2377[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2378[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2379[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2380[{'id': 5, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-10-03T07:06:30.051Z', 'started_at': None, 'finished_at': '2026-10-03T07:06:40.002Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'e874471c6bba3c76d684eaabbff376e5652e08d0', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-10-03T07:06:40.178Z', 'started_at': None, 'finished_at': '2026-10-03T07:06:40.335Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]