vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #395
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.cnZ4QTkpgQ', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: 599670e8-9a9c-4d4f-8d86-19d0a564670016gitlab # Superblock backups stored on blocks:17(finished: start all VMs, in 74.20 seconds)18gitlab # 32768, 98304, 163840, 22937619gitlab: waiting for unit setup-gitlab.service20gitlab # 21gitlab: waiting for the VM to finish booting22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1m222qal5hixbmwgck0hvmkayxdbn9xj-nixos-system-gitlab-test/init regInfo=/nix/store/r437ra32w36adghmx664hmbd9yqya0v6-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/r437ra32w36adghmx664hmbd9yqya0v6-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000030] arm-pv: using stolen time PV108gitlab # [ 0.000446] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000652] Console: colour dummy device 80x25110gitlab # [ 0.000659] printk: legacy console [tty0] enabled111gitlab # [ 0.000846] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000852] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000935] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.001066] landlock: Up and running.115gitlab # [ 0.001070] Yama: becoming mindful.116gitlab # [ 0.001531] LSM support for eBPF active117gitlab # [ 0.001734] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001845] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.003196] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003945] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003950] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.004133] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.005161] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.005253] EFI services will not be available.125gitlab # [ 0.005438] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005968] Detected PIPT I-cache on CPU1127gitlab # [ 0.006032] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.006106] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.006180] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007055] Detected PIPT I-cache on CPU2131gitlab # [ 0.007123] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007195] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007270] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008034] Detected PIPT I-cache on CPU3135gitlab # [ 0.008107] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008180] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.008253] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.008555] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.008581] SMP: Total of 4 processors activated.140gitlab # [ 0.008584] CPU: All CPU(s) started at EL1141gitlab # [ 0.008592] CPU features: detected: Branch Target Identification142gitlab # [ 0.008596] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.008599] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.008602] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.008606] CPU features: detected: Common not Private translations146gitlab # [ 0.008608] CPU features: detected: CRC32 instructions147gitlab # [ 0.008611] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.008614] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.008617] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.008620] CPU features: detected: E0PD151gitlab # [ 0.008622] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.008625] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.008628] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.008631] CPU features: detected: Fine Grained Traps155gitlab # [ 0.008634] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.008638] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.008640] CPU features: detected: LSE atomic instructions158gitlab # [ 0.008643] CPU features: detected: Privileged Access Never159gitlab # [ 0.008645] CPU features: detected: PMUv3160gitlab # [ 0.008648] CPU features: detected: RAS Extension Support161gitlab # [ 0.008650] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.008652] CPU features: detected: Random Number Generator163gitlab # [ 0.008655] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.008657] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.008660] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.008663] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.008752] alternatives: applying system-wide alternatives168gitlab # [ 0.011763] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.011998] Memory: 5963784K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.013567] devtmpfs: initialized171gitlab # [ 0.017071] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.017139] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.017321] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.017326] 0 pages in range for non-PLT usage175gitlab # [ 0.017327] 508288 pages in range for PLT usage176gitlab # [ 0.017453] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.018381] DMI not present or invalid.178gitlab # [ 0.021927] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.024578] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.024844] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.026819] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.026842] audit: initializing netlink subsys (disabled)183gitlab # [ 0.027017] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.027495] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.027497] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.027501] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.027504] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.027507] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.027565] cpuidle: using governor ladder190gitlab # [ 0.027584] cpuidle: using governor menu191gitlab # [ 0.027790] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.027913] ASID allocator initialised with 65536 entries193gitlab # [ 0.029090] Serial: AMBA PL011 UART driver194gitlab # [ 0.034359] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.034553] printk: console [ttyAMA0] enabled196gitlab # [ 0.040356] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.040369] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.040374] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.040379] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.040383] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.040387] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.040392] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.040396] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.047411] fbcon: Taking over console205gitlab # [ 0.047423] ACPI: Interpreter disabled.206gitlab # [ 0.048371] iommu: Default domain type: Translated207gitlab # [ 0.048381] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.062527] SCSI subsystem initialized209gitlab # [ 0.063082] usbcore: registered new interface driver usbfs210gitlab # [ 0.063136] usbcore: registered new interface driver hub211gitlab # [ 0.063163] usbcore: registered new device driver usb212gitlab # [ 0.063666] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.063672] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.063700] PTP clock support registered215gitlab # [ 0.063828] EDAC MC: Ver: 3.0.0216gitlab # [ 0.064233] scmi_core: SCMI protocol bus registered217gitlab # [ 0.064869] FPGA manager framework218gitlab # [ 0.066154] vgaarb: loaded219gitlab # [ 0.066978] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.070337] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.070375] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.070796] netfs: FS-Cache loaded223gitlab # [ 0.070924] pnp: PnP ACPI: disabled224gitlab # [ 0.076534] NET: Registered PF_INET protocol family225gitlab # [ 0.078019] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.120010] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.120077] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.120138] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.120568] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.121408] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.121529] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.121604] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.121769] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.122067] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.122096] NET: Registered PF_XDP protocol family236gitlab # [ 0.122114] PCI: CLS 0 bytes, default 64237gitlab # [ 0.122339] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.127152] kvm [1]: HYP mode not available239gitlab # [ 0.184104] Initialise system trusted keyrings240gitlab # [ 0.184299] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.185011] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.185102] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.204155] Key type asymmetric registered244gitlab # [ 0.204162] Asymmetric key parser 'x509' registered245gitlab # [ 0.204231] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.204406] io scheduler mq-deadline registered247gitlab # [ 0.204415] io scheduler kyber registered248gitlab # [ 0.216703] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.223108] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.223656] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.223678] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.223726] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.223737] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.223772] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.223803] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.223985] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.223995] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.224001] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.224006] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.224011] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.224181] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.225006] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.225364] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.225385] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.225429] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.225449] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.226309] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.226652] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.226671] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.226720] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.227625] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.227969] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.227988] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.228031] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.228869] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.229208] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.229227] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.229276] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.230107] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.230450] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.230470] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.230513] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.231363] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.231691] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.231710] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.231753] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.232561] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.232892] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.232917] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.232959] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.232984] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.233901] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.234257] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.234300] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.235157] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.235516] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.235561] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.236255] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.236595] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.237051] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.237477] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.237496] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.237539] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.238349] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.238695] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.238713] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.238757] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.239834] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.239845] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.239851] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.239920] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.240002] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.240079] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.240161] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.240243] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.240322] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.240415] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.240493] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.240570] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.240675] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.240765] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.240794] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.240823] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.240851] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.240881] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.240909] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.240939] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.240973] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.241008] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.241038] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.241070] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.241101] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.241130] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.241158] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.241193] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.241228] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.241260] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.241298] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.241327] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.241355] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.241384] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.241426] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.241434] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.241439] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.243697] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.262175] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.265627] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.269110] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.272809] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.277506] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.280653] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.283907] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.289024] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.292361] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.295409] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.299536] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.308208] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.311045] msm_serial: driver initialized359gitlab # [ 0.311285] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.311359] STM32 USART driver initialized361gitlab # [ 0.348200] loop: module loaded362gitlab # [ 0.348535] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.351699] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.354936] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.357808] virtio_blk virtio5: [vdb] 12394704 512-byte logical blocks (6.35 GB/5.91 GiB)366gitlab # [ 0.360655] megasas: 07.734.00.00-rc1367gitlab # [ 0.361718] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.364896] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.364922] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.367589] Using buffer write method371gitlab # [ 0.367677] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.370417] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.370472] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.373775] Using buffer write method375gitlab # [ 0.373804] Concatenating MTD devices:376gitlab # [ 0.373808] (0): "0.flash"377gitlab # [ 0.373811] (1): "0.flash"378gitlab # [ 0.373814] into device "0.flash"379gitlab # [ 0.572778] Freeing initrd memory: 27184K380gitlab # [ 0.583292] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.588385] thunder_xcv, ver 1.0382gitlab # [ 0.588479] thunder_bgx, ver 1.0383gitlab # [ 0.588512] nicpf, ver 1.0384gitlab # [ 0.589434] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.589441] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.589489] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.589498] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.589549] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.589554] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.589588] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.589593] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.589954] sky2: driver version 1.30393gitlab # [ 0.592552] usbcore: registered new interface driver usb-storage394gitlab # [ 0.592805] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.592827] usbserial: USB Serial support registered for generic396gitlab # [ 0.593005] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.593067] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.593482] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.593785] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.595425] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.595465] rtc-pl031 9010000.pl031: setting system clock to 2026-10-02T14:09:18 UTC (1790950158)402gitlab # [ 0.595979] i2c_dev: i2c /dev entries driver403gitlab # [ 0.600446] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.600454] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.600868] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.601459] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.604013] hid: raw HID events driver (C) Jiri Kosina408gitlab # [ 0.604536] usbcore: registered new interface driver usbhid409gitlab # [ 0.604542] usbhid: USB HID core driver410gitlab # [ 0.607087] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00411gitlab # [ 0.607830] hub 1-0:1.0: USB hub found412gitlab # [ 0.607870] hub 1-0:1.0: 6 ports detected413gitlab # [ 0.706613] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.709504] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.709958] NET: Registered PF_INET6 protocol family416gitlab # [ 0.711887] Segment Routing with IPv6417gitlab # [ 0.711913] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.712053] NET: Registered PF_PACKET protocol family419gitlab # [ 0.712299] 9pnet: Installing 9P2000 support420gitlab # [ 0.715151] Key type dns_resolver registered421gitlab # [ 0.723821] registered taskstats version 1422gitlab # [ 0.724099] Loading compiled-in X.509 certificates423gitlab # [ 0.735561] Demotion targets for Node 0: null424gitlab # [ 0.735768] Key type .fscrypt registered425gitlab # [ 0.735775] Key type fscrypt-provisioning registered426gitlab # [ 0.735958] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.735988] ima: Allocated hash algorithm: sha1428gitlab # [ 0.736031] ima: No architecture policies found429gitlab # [ 0.737103] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.760813] clk: Disabling unused clocks431gitlab # [ 0.760850] PM: genpd: Disabling unused power domains432gitlab # [ 0.765481] Freeing unused kernel memory: 4736K433gitlab # [ 0.765745] Run /init as init process434gitlab # [ 0.794873] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.927026] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.076568] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.129161] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.129200] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.129242] systemd[1]: Detected architecture arm64.440gitlab # [ 1.129251] systemd[1]: Running in initrd.441gitlab # [ 1.129699] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.129938] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.195381] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.315023] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.324516] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.391903] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.418257] systemd[1]: Created slice Slice /system/modprobe.448gitlab # [ 1.418554] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.449gitlab # [ 1.418591] systemd[1]: Expecting device /dev/disk/by-label/nix-store...450gitlab # [ 1.418606] systemd[1]: Expecting device /dev/disk/by-label/nixos...451gitlab # [ 1.418630] systemd[1]: Reached target Path Units.452gitlab # [ 1.418649] systemd[1]: Reached target Slice Units.453gitlab # [ 1.418668] systemd[1]: Reached target Swaps.454gitlab # [ 1.418686] systemd[1]: Reached target Timer Units.455gitlab # [ 1.418879] systemd[1]: Listening on D-Bus System Message Bus Socket.456gitlab # [ 1.419201] systemd[1]: Listening on Journal Socket (/dev/log).457gitlab # [ 1.419381] systemd[1]: Listening on Journal Sockets.458gitlab # [ 1.419542] systemd[1]: Listening on udev Control Socket.459gitlab # [ 1.419696] systemd[1]: Listening on udev Kernel Socket.460gitlab # [ 1.419719] systemd[1]: Reached target Socket Units.461gitlab # [ 1.421623] systemd[1]: Starting Create List of Static Device Nodes...462gitlab # [ 1.423481] systemd[1]: Starting Load Kernel Module 9pnet_virtio...463gitlab # [ 1.423560] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs464gitlab # [ 1.425408] systemd[1]: Mounting Kernel Configuration File System...465gitlab # [ 1.443183] systemd[1]: Starting Journal Service...466gitlab # [ 1.445590] systemd[1]: Starting Load Kernel Modules...467gitlab # [ 1.445745] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os468gitlab # [ 1.447879] systemd[1]: Starting Coldplug All udev Devices...469gitlab # [ 1.449924] systemd[1]: Finished Create List of Static Device Nodes.470gitlab # [ 1.454089] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.471gitlab # [ 1.456267] systemd[1]: Finished Load Kernel Module 9pnet_virtio.472gitlab # [ 1.458160] systemd[1]: Mounted Kernel Configuration File System.473gitlab # [ 1.463195] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...474gitlab # [ 1.477434] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0475gitlab # [ 1.477645] [drm] features: -virgl +edid -resource_blob -host_visible476gitlab # [ 1.477658] [drm] features: -context_init477gitlab # [ 1.478959] [drm] number of scanouts: 1478gitlab # [ 1.479231] systemd-journald[95]: Collecting audit messages is disabled.479gitlab # [ 1.483268] [drm] number of cap sets: 0480gitlab # [ 1.483505] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.481gitlab # [ 1.484114] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev482gitlab # [ 1.484847] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.483gitlab # [ 1.490615] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2484gitlab # [ 1.490783] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0485gitlab # [ 1.496782] systemd[1]: Starting Create Static Device Nodes in /dev...486gitlab # [ 1.499543] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic487gitlab # [ 1.499560] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0488gitlab # [ 1.510842] Console: switching to colour frame buffer device 160x50489gitlab # [ 1.512323] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device490gitlab # [ 1.536169] systemd[1]: Finished Load Kernel Modules.491gitlab # [ 1.538513] systemd[1]: Starting Apply Kernel Variables...492gitlab # [ 1.539591] systemd-modules-load[96]: Using 4 probe threads493gitlab # [ 1.542095] systemd[1]: Started Journal Service.494gitlab # [ 1.542073] systemd-modules-load[96]: Module 'virtio_balloon' is built in495gitlab # [ 1.544763] systemd-modules-load[96]: Module 'virtio_console' is built in496gitlab # [ 1.546082] systemd-modules-load[96]: Module 'virtio_rng' is built in497gitlab # [ 1.548053] systemd-modules-load[96]: Inserted module 'dm_mod'498gitlab # [ 1.552646] systemd-modules-load[96]: Inserted module 'virtio_gpu'499gitlab # [ 1.554393] systemd[1]: Finished Create Static Device Nodes in /dev.500gitlab # [ 1.556151] systemd[1]: Reached target Preparation for Local File Systems.501gitlab # [ 1.557954] systemd[1]: Reached target Local File Systems.502gitlab # [ 1.559342] systemd[1]: Starting Create System Files and Directories...503gitlab # [ 1.561079] systemd[1]: Starting Rule-based Manager for Device Events and Files...504gitlab # [ 1.562773] systemd[1]: Finished Apply Kernel Variables.505gitlab # [ 1.577927] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.589936] systemd-udevd[112]: Using default interface naming scheme 'v261'.507gitlab # [ 1.603107] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.623045] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.661522] systemd-vconsole-setup[130]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.663593] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.101188] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.102705] systemd[1]: Reached target System Initialization.513gitlab # [ 2.104148] systemd[1]: Reached target Basic System.514gitlab # [ 2.219000] (udev-worker)[136]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.224446] (udev-worker)[136]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.251295] systemd[1]: Found device /dev/disk/by-label/nix-store.517gitlab # [ 2.266840] (udev-worker)[144]: Network interface NamePolicy= disabled on kernel command line.518gitlab # [ 2.270192] systemd[1]: Found device /dev/disk/by-label/nixos.519gitlab # [ 2.274210] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.275915] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.310931] systemd-fsck[152]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.314687] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.448129] systemd[1]: Mounting /sysroot...524gitlab # [ 2.482493] EXT4-fs (vda): mounted filesystem 599670e8-9a9c-4d4f-8d86-19d0a5646700 r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.483698] systemd[1]: Mounted /sysroot.526gitlab # [ 2.485149] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.489440] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.490521] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.504772] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.509926] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.520479] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.552209] erofs (device vdb): mounted with root inode @ nid 128.533gitlab # [ 2.552833] systemd[1]: Starting Mountpoints Configured in the Real Root...534gitlab # [ 2.567268] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload.535gitlab # [ 2.571000] systemd[1]: Mounted /sysroot/nix/.ro-store.536gitlab # [ 2.572420] systemd[1]: Mounted /sysroot/nix/.rw-store.537gitlab # [ 2.573511] systemd[1]: Mounted /sysroot/run.538gitlab # [ 2.599367] systemd[1]: Starting rw-sysroot-nix-store.service...539gitlab # [ 2.611221] systemd[1]: Mounted /sysroot/tmp/shared.540gitlab # [ 2.616442] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)...541gitlab # [ 2.618832] systemd[1]: Reloading...542gitlab # [ 2.726715] systemd[1]: Reloading finished in 107 ms.543gitlab # [ 2.784401] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace...544gitlab # [ 2.787064] systemd[1]: Mounted /sysroot/tmp/xchg.545gitlab # [ 2.789193] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.546gitlab # [ 2.792248] systemd[1]: Finished rw-sysroot-nix-store.service.547gitlab # [ 2.796217] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace...548gitlab # [ 2.799532] (udev-worker)[139]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory549gitlab # [ 2.802666] (udev-worker)[139]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory550gitlab # [ 2.808189] systemd[1]: initrd-parse-etc.service: Deactivated successfully.551gitlab # [ 2.809281] systemd[1]: Finished Mountpoints Configured in the Real Root.552gitlab # [ 2.810278] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.553gitlab # [ 2.811421] systemd[1]: Starting rw-sysroot-nix-store.service...554gitlab # [ 2.812384] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.555gitlab # [ 2.813438] systemd[1]: Stopped Virtual Console Setup.556gitlab # [ 2.814202] systemd[1]: Stopping Virtual Console Setup...557gitlab # [ 2.815001] systemd[1]: Starting Virtual Console Setup...558gitlab # [ 2.829030] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.559gitlab # [ 2.830210] systemd[1]: Stopped Virtual Console Setup.560gitlab # [ 2.831097] systemd[1]: Starting Virtual Console Setup...561gitlab # [ 2.835223] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.562gitlab # [ 2.836390] systemd[1]: Finished rw-sysroot-nix-store.service.563gitlab # [ 2.853883] systemd-vconsole-setup[197]: Configuration of first virtual console was skipped, ignoring remaining ones.564gitlab # [ 2.855775] systemd[1]: Finished Virtual Console Setup.565gitlab # [ 3.450878] systemd[1]: Mounting /sysroot/nix/store...566gitlab # [ 3.495685] systemd[1]: Mounted /sysroot/nix/store.567gitlab # [ 3.497223] systemd[1]: Reached target Initrd File Systems.568gitlab # [ 3.498619] systemd[1]: Starting Find NixOS closure...569gitlab # [ 3.500467] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...570gitlab # [ 3.537678] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.571gitlab # [ 3.539795] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.572gitlab # [ 3.548947] systemd[1]: Finished Find NixOS closure.573gitlab # [ 3.550234] systemd[1]: Reached target Initrd Default Target.574gitlab # [ 3.551635] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...575gitlab # [ 3.585631] systemd[1]: Stopped target Initrd Default Target.576gitlab # [ 3.587049] systemd[1]: Stopped target Basic System.577gitlab # [ 3.588319] systemd[1]: Stopped target Initrd Root Device.578gitlab # [ 3.589611] systemd[1]: Stopped target Path Units.579gitlab # [ 3.590780] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.580gitlab # [ 3.592605] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.581gitlab # [ 3.594508] systemd[1]: Stopped target Slice Units.582gitlab # [ 3.595690] systemd[1]: Stopped target Socket Units.583gitlab # [ 3.596955] systemd[1]: Stopped target System Initialization.584gitlab # [ 3.598299] systemd[1]: Stopped target Swaps.585gitlab # [ 3.599387] systemd[1]: Stopped target Timer Units.586gitlab # [ 3.600628] systemd[1]: dbus.socket: Deactivated successfully.587gitlab # [ 3.602003] systemd[1]: Closed D-Bus System Message Bus Socket.588gitlab # [ 3.604765] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.589gitlab # [ 3.607714] systemd[1]: Stopped Find NixOS closure.590gitlab # [ 3.611209] systemd[1]: Starting Load Kernel Module 9pnet_virtio...591gitlab # [ 3.612820] systemd[1]: Starting rw-sysroot-nix-store.service...592gitlab # [ 3.614196] systemd[1]: systemd-sysctl.service: Deactivated successfully.593gitlab # [ 3.615818] systemd[1]: Stopped Apply Kernel Variables.594gitlab # [ 3.617448] systemd[1]: systemd-modules-load.service: Deactivated successfully.595gitlab # [ 3.619296] systemd[1]: Stopped Load Kernel Modules.596gitlab # [ 3.620552] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.597gitlab # [ 3.622366] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.598gitlab # [ 3.624399] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.599gitlab # [ 3.626197] systemd[1]: Stopped Create System Files and Directories.600gitlab # [ 3.627743] systemd[1]: Stopped target Local File Systems.601gitlab # [ 3.629233] systemd[1]: Stopped target Preparation for Local File Systems.602gitlab # [ 3.630889] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.603gitlab # [ 3.632608] systemd[1]: Stopped Coldplug All udev Devices.604gitlab # [ 3.634043] systemd[1]: Stopping Rule-based Manager for Device Events and Files...605gitlab # [ 3.635794] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.606gitlab # [ 3.637456] systemd[1]: Stopped Virtual Console Setup.607gitlab # [ 3.638583] systemd[1]: initrd-cleanup.service: Deactivated successfully.608gitlab # [ 3.639993] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.609gitlab # [ 3.641421] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.610gitlab # [ 3.642954] systemd[1]: Finished rw-sysroot-nix-store.service.611gitlab # [ 3.644241] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.612gitlab # [ 3.645654] systemd[1]: Finished Load Kernel Module 9pnet_virtio.613gitlab # [ 3.646906] systemd[1]: systemd-udevd.service: Deactivated successfully.614gitlab # [ 3.648433] systemd[1]: Stopped Rule-based Manager for Device Events and Files.615gitlab # [ 3.649790] systemd[1]: systemd-udevd.service: Consumed 1.998s CPU time over 2.075s wall clock time, 31.3M memory peak.616gitlab # [ 3.651680] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.617gitlab # [ 3.653083] systemd[1]: Closed udev Control Socket.618gitlab # [ 3.654076] systemd[1]: Starting Cleanup udev Database...619gitlab # [ 3.655143] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.620gitlab # [ 3.656595] systemd[1]: Stopped Create Static Device Nodes in /dev.621gitlab # [ 3.657761] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.622gitlab # [ 3.659350] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.623gitlab # [ 3.660671] systemd[1]: kmod-static-nodes.service: Deactivated successfully.624gitlab # [ 3.661857] systemd[1]: Stopped Create List of Static Device Nodes.625gitlab # [ 3.696313] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.626gitlab # [ 3.698191] systemd[1]: Finished Cleanup udev Database.627gitlab # [ 3.699476] systemd[1]: Reached target Switch Root.628gitlab # [ 3.700730] systemd[1]: Starting NixOS Activation...629gitlab # [ 3.834347] initrd-nixos-activation-start[224]: booting system configuration /nix/store/1m222qal5hixbmwgck0hvmkayxdbn9xj-nixos-system-gitlab-test630gitlab # [ 3.872278] initrd-nixos-activation-start[224]: running activation script...631gitlab # [ 4.117716] initrd-nixos-activation-start[249]: setting up /etc...632gitlab # [ 4.233840] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.633gitlab # [ 4.235780] systemd[1]: Finished NixOS Activation.634gitlab # [ 4.237125] systemd[1]: Starting Switch Root...635gitlab # [ 4.268723] systemd[1]: Switching root.636gitlab # [ 4.398127] systemd-journald[95]: Received SIGTERM from PID 1 (systemd).637gitlab # [ 4.956404] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)638gitlab # [ 4.956492] systemd[1]: Detected virtualization qemu.639gitlab # [ 4.956559] systemd[1]: Detected architecture arm64.640gitlab # [ 4.956683] systemd[1]: Detected first boot.641gitlab # [ 4.957573] systemd[1]: Initializing machine ID from random generator.642gitlab # [ 5.162211] systemd[1]: bpf-restrict-fs: LSM BPF program attached643gitlab # [ 5.245403] systemd[1]: Applying preset policy.644gitlab # [ 5.405497] systemd[1]: Populated /etc with preset unit settings.645gitlab # [ 5.557194] systemd[1]: initrd-switch-root.service: Deactivated successfully.646gitlab # [ 5.557676] systemd[1]: Stopped initrd-switch-root.service.647gitlab # [ 5.560171] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.648gitlab # [ 5.563874] systemd[1]: Created slice Slice /system/getty.649gitlab # [ 5.566638] systemd[1]: Created slice GitLab DevOps Platform Slice.650gitlab # [ 5.569112] systemd[1]: Created slice User and Session Slice.651gitlab # [ 5.569379] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.652gitlab # [ 5.570221] systemd[1]: Started Forward Password Requests to Wall Directory Watch.653gitlab # [ 5.570934] systemd[1]: Expecting device /dev/hvc0...654gitlab # [ 5.575120] systemd[1]: Expecting device /dev/ttyAMA0...655gitlab # [ 5.575686] systemd[1]: Reached target Local Encrypted Volumes.656gitlab # [ 5.576109] systemd[1]: Stopped target initrd-fs.target.657gitlab # [ 5.576494] systemd[1]: Stopped target initrd-root-fs.target.658gitlab # [ 5.576882] systemd[1]: Stopped target initrd-switch-root.target.659gitlab # [ 5.577267] systemd[1]: Reached target Virtual Machines and Containers.660gitlab # [ 5.577673] systemd[1]: Reached target Path Units.661gitlab # [ 5.578078] systemd[1]: Reached target Remote File Systems.662gitlab # [ 5.578447] systemd[1]: Reached target Slice Units.663gitlab # [ 5.578825] systemd[1]: Reached target Swaps.664gitlab # [ 5.581034] systemd[1]: Listening on Query the User Interactively for a Password.665gitlab # [ 5.583410] systemd[1]: Listening on Process Core Dump Socket.666gitlab # [ 5.585104] systemd[1]: Listening on Credential Encryption/Decryption.667gitlab # [ 5.587104] systemd[1]: Listening on Factory Reset Management.668gitlab # [ 5.587905] systemd[1]: Listening on Hostname Service Socket.669gitlab # [ 5.592942] systemd[1]: Starting Journal Log Access Socket...670gitlab # [ 5.594453] systemd[1]: Listening on Journal Audit Socket.671gitlab # [ 5.596369] systemd[1]: Listening on Console Output Muting Service Socket.672gitlab # [ 5.597114] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.673gitlab # [ 5.597650] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os674gitlab # [ 5.598294] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki675gitlab # [ 5.600859] systemd[1]: Listening on Disk Repartitioning Service Socket.676gitlab # [ 5.601451] systemd[1]: Listening on udev Control Socket.677gitlab # [ 5.602053] systemd[1]: Listening on udev Varlink Socket.678gitlab # [ 5.607070] systemd[1]: Mounting Huge Pages File System...679gitlab # [ 5.611684] systemd[1]: Mounting POSIX Message Queue File System...680gitlab # [ 5.618390] systemd[1]: Mounting Kernel Debug File System...681gitlab # [ 5.624621] systemd[1]: Mounting Kernel Trace File System...682gitlab # [ 5.628174] systemd[1]: Starting Create List of Static Device Nodes...683gitlab # [ 5.637602] systemd[1]: Starting Load Kernel Module 9pnet_virtio...684gitlab # [ 5.638410] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs685gitlab # [ 5.649209] systemd[1]: Mounting Kernel Configuration File System...686gitlab # [ 5.649840] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm687gitlab # [ 5.650412] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore688gitlab # [ 5.659268] systemd[1]: Starting Load Kernel Module fuse...689gitlab # [ 5.661222] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67690gitlab # [ 5.667901] systemd[1]: Starting Journal Service...691gitlab # [ 5.671294] systemd[1]: Starting Load Kernel Modules...692gitlab # [ 5.678636] fuse: init (API version 7.45)693gitlab # [ 5.679758] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...694gitlab # [ 5.693736] systemd[1]: Starting Remount Root and Kernel File Systems...695gitlab # [ 5.694494] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os696gitlab # [ 5.701271] systemd[1]: Starting Coldplug All udev Devices...697gitlab # [ 5.705304] systemd[1]: Listening on Journal Log Access Socket.698gitlab # [ 5.708778] systemd[1]: Mounted Huge Pages File System.699gitlab # [ 5.709685] systemd[1]: Mounted POSIX Message Queue File System.700gitlab # [ 5.710454] systemd[1]: Mounted Kernel Debug File System.701gitlab # [ 5.718757] systemd[1]: Mounted Kernel Trace File System.702gitlab # [ 5.722158] systemd[1]: Finished Create List of Static Device Nodes.703gitlab # [ 5.723400] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.704gitlab # [ 5.724820] systemd[1]: Finished Load Kernel Module 9pnet_virtio.705gitlab # [ 5.725774] systemd[1]: Mounted Kernel Configuration File System.706gitlab # [ 5.726796] systemd[1]: modprobe@fuse.service: Deactivated successfully.707gitlab # [ 5.727878] systemd[1]: Finished Load Kernel Module fuse.708gitlab # [ 5.734577] systemd[1]: Mounting FUSE Control File System...709gitlab # [ 5.738923] systemd-journald[321]: Collecting audit messages is enabled.710gitlab # [ 5.739227] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...711gitlab # [ 5.755184] EXT4-fs (vda): re-mounted 599670e8-9a9c-4d4f-8d86-19d0a5646700.712gitlab # [ 5.758124] systemd[1]: Queued start job for default target Multi-User System.713gitlab # [ 5.762626] systemd[1]: Finished Load Kernel Modules.714gitlab # [ 5.761855] systemd[1]: Unnecessary job was removed for /dev/vdb.715gitlab # [ 5.765023] systemd[1]: Started Journal Service.716gitlab # [ 5.764653] systemd[1]: systemd-journald.service: Deactivated successfully.717gitlab # [ 5.766321] systemd-modules-load[322]: Using 3 probe threads718gitlab # [ 5.767573] systemd-modules-load[322]: Module 'atkbd' is built in719gitlab # [ 5.768914] systemd-modules-load[322]: Module 'loop' is built in720gitlab # [ 5.770086] systemd-modules-load[322]: Inserted module 'tls'721gitlab # [ 5.771844] systemd[1]: Finished Remount Root and Kernel File Systems.722gitlab # [ 5.776515] systemd[1]: Mounted FUSE Control File System.723gitlab # [ 5.782520] systemd[1]: Listening on Disk Image Download Service Socket.724gitlab # [ 5.787535] systemd[1]: Starting Firewall...725gitlab # [ 5.789397] systemd-oomd[323]: No swap; memory pressure usage will be degraded726gitlab # [ 5.796564] systemd[1]: Starting Flush Journal to Persistent Storage...727gitlab # [ 5.797919] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore728gitlab # [ 5.802260] systemd[1]: Starting Load/Save OS Random Seed...729gitlab # [ 5.808434] systemd[1]: Starting Apply Kernel Variables...730gitlab # [ 5.811557] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os731gitlab # [ 5.813546] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.732gitlab # [ 5.814998] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.733gitlab # [ 5.822866] systemd-journald[321]: Received client request to flush runtime journal.734gitlab # [ 5.846546] systemd[1]: Starting Create Static Device Nodes in /dev...735gitlab # [ 5.848092] systemd[1]: Finished Load/Save OS Random Seed.736gitlab # [ 5.852621] systemd[1]: Reached target First Boot Complete.737gitlab # [ 5.856579] systemd[1]: Finished Flush Journal to Persistent Storage.738gitlab # [ 5.858305] systemd[1]: Finished Apply Kernel Variables.739gitlab # [ 5.859460] systemd[1]: Finished Create Static Device Nodes in /dev.740gitlab # [ 5.862929] systemd[1]: Reached target Preparation for Local File Systems.741gitlab # [ 5.864546] systemd[1]: Starting Rule-based Manager for Device Events and Files...742gitlab # [ 5.929002] systemd-udevd[361]: Using default interface naming scheme 'v261'.743gitlab # [ 5.956876] systemd[1]: Started Rule-based Manager for Device Events and Files.744gitlab # [ 6.175598] systemd[1]: Finished Firewall.745gitlab # [ 6.356996] systemd[1]: Finished Coldplug All udev Devices.746gitlab # [ 6.396658] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs747gitlab # [ 6.442748] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.748gitlab # [ 6.462546] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.749gitlab # [ 6.468990] (udev-worker)[407]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.750gitlab # [ 6.472532] (udev-worker)[407]: Network interface NamePolicy= disabled on kernel command line.751gitlab # [ 6.489319] systemd[1]: Condition check resulted in Virtio network device being skipped.752gitlab # [ 6.491362] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore753gitlab # [ 6.493725] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67754gitlab # [ 6.496714] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore755gitlab # [ 6.499353] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os756gitlab # [ 6.501150] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os757gitlab # [ 6.503073] (udev-worker)[406]: Network interface NamePolicy= disabled on kernel command line.758gitlab # [ 6.559813] mousedev: PS/2 mouse device common for all mice759gitlab # [ 6.557491] systemd[1]: Mounting /run/wrappers...760gitlab # [ 6.579230] systemd[1]: Mounted /run/wrappers.761gitlab # [ 6.580194] systemd[1]: Reached target Local File Systems.762gitlab # [ 6.581923] systemd[1]: Listening on Boot Loader Control Service Socket.763gitlab # [ 6.583032] systemd[1]: Starting register-nix-paths.service...764gitlab # [ 6.586344] systemd[1]: Starting Create SUID/SGID Wrappers...765gitlab # [ 6.588281] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.766gitlab # [ 6.592590] systemd[1]: Starting Save Transient machine-id to Disk...767gitlab # [ 6.598744] systemd[1]: Starting Create System Files and Directories...768gitlab # [ 6.640304] systemd[1]: Finished Create System Files and Directories.769gitlab # [ 6.644870] systemd[1]: Starting Rebuild Journal Catalog...770gitlab # [ 6.659164] systemd[1]: Starting Record System Boot/Shutdown in UTMP...771gitlab # [ 6.660604] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.772gitlab # [ 6.669709] systemd[1]: Finished Save Transient machine-id to Disk.773gitlab # [ 6.685562] systemd[1]: Finished Record System Boot/Shutdown in UTMP.774gitlab # [ 6.696643] systemd[1]: Finished Rebuild Journal Catalog.775gitlab # [ 6.700214] systemd[1]: Starting Update is Completed...776gitlab # [ 6.727734] systemd[1]: Finished Update is Completed.777gitlab # [ 6.786491] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.778gitlab # [ 6.787923] systemd[1]: Finished Create SUID/SGID Wrappers.779gitlab # [ 6.789371] systemd[1]: suid-sgid-wrappers.service: Consumed 138ms CPU time over 200ms wall clock time, 6.5M memory peak, 1.9M read from disk.780gitlab # [ 7.054376] systemd[1]: Finished register-nix-paths.service.781gitlab # [ 7.056521] systemd[1]: Reached target System Initialization.782gitlab # [ 7.058648] systemd[1]: Started Discard unused filesystem blocks once a week.783gitlab # [ 7.060475] systemd[1]: Started logrotate.timer.784gitlab # [ 7.061303] systemd[1]: Started Daily Cleanup of Temporary Directories.785gitlab # [ 7.062341] systemd[1]: Reached target Timer Units.786gitlab # [ 7.063150] systemd[1]: Listening on D-Bus System Message Bus Socket.787gitlab # [ 7.064252] systemd[1]: Listening on Nix Daemon Socket.788gitlab # [ 7.170112] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3789gitlab # [ 7.177867] systemd[1]: Starting nixbot CI service socket...790gitlab # [ 7.180227] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.791gitlab # [ 7.184336] systemd[1]: Listening on nixbot CI service socket.792gitlab # [ 7.187183] systemd[1]: Reached target Socket Units.793gitlab # [ 7.192110] systemd[1]: Starting D-Bus System Message Bus...794gitlab # [ 7.229849] systemd[1]: Starting Virtual Console Setup...795gitlab # [ 7.235247] dbus-broker-launch[569]: Looking up NSS user entry for 'systemd-timesync'...796gitlab # [ 7.236888] dbus-broker-launch[569]: NSS returned no entry for 'systemd-timesync'797gitlab # [ 7.238240] dbus-broker-launch[569]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"798gitlab # [ 7.241705] systemd[1]: Started D-Bus System Message Bus.799gitlab # [ 7.253573] dbus-broker-launch[569]: Ready800gitlab # [ 7.254667] systemd[1]: Reached target Basic System.801gitlab # [ 7.257499] systemd[1]: Started backdoor.service.802gitlab # [ 7.259720] systemd[1]: Starting gitlab-config.service...803gitlab # [ 7.278683] systemd-vconsole-setup[573]: Configuration of first virtual console was skipped, ignoring remaining ones.804gitlab # [ 7.288469] systemd[1]: Starting Import lastlog data into lastlog2 database...805gitlab # [ 7.296310] systemd[1]: Starting Logrotate configuration check...806gitlab # [ 7.302602] systemd[1]: Starting Name Service Cache Daemon (nsncd)...807gitlab # connecting to host...808gitlab # [ 7.307436] systemd[1]: Starting Post-Boot Actions...809gitlab # [ 7.309189] logrotate[579]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.810gitlab # [ 7.314785] logrotate[579]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf811gitlab # [ 7.316805] logrotate[579]: note: 'monthly' overrides previously specified 'weekly'812gitlab # [ 7.318053] logrotate[579]: Reading state from file: /var/lib/logrotate.status813gitlab # [ 7.319238] logrotate[579]: state file /var/lib/logrotate.status does not exist814gitlab # [ 7.321072] logrotate[579]: Allocating hash table for state file, size 64 entries815gitlab # [ 7.323673] logrotate[579]: Handling 4 logs816gitlab # [ 7.326978] logrotate[579]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed817gitlab # [ 7.329746] logrotate[579]: considering log /var/log/btmp818gitlab: Guest shell says: b'Spawning backdoor root shell...\n'819gitlab # [ 7.330629] logrotate[579]: Creating new state820gitlab # [ 7.332030] logrotate[579]: Now: 2026-10-02 14:09821gitlab # [ 7.332799] logrotate[579]: Last rotated at 2026-10-02 14:00822gitlab # [ 7.333646] logrotate[579]: log does not need rotating (log has already been rotated)823gitlab: connected to guest root shell824gitlab # [ 7.334794] logrotate[579]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed825gitlab: (connecting took 7.80 seconds)826gitlab # [ 7.337084] logrotate[579]: considering log /var/log/wtmp827gitlab: (finished: waiting for the VM to finish booting, in 7.80 seconds)828gitlab # [ 7.337950] logrotate[579]: Creating new state829gitlab # [ 7.338615] logrotate[579]: Now: 2026-10-02 14:09830gitlab # [ 7.339401] logrotate[579]: Last rotated at 2026-10-02 14:00831gitlab # [ 7.340299] logrotate[579]: log does not need rotating (log has already been rotated)832gitlab # [ 7.341408] logrotate[579]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed833gitlab # [ 7.344696] logrotate[579]: switching euid from 0 to 165 and egid from 0 to 165 (pid 579)834gitlab # [ 7.345910] logrotate[579]: considering log /var/gitlab/state/log/*.log835gitlab # [ 7.346890] logrotate[579]: log /var/gitlab/state/log/*.log does not exist -- skipping836gitlab # [ 7.348137] logrotate[579]: Creating new state837gitlab # [ 7.348835] logrotate[579]: switching euid from 165 to 0 and egid from 165 to 0 (pid 579)838gitlab # [ 7.350039] logrotate[579]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed839gitlab # [ 7.351778] logrotate[579]: switching euid from 0 to 60 and egid from 0 to 60 (pid 579)840gitlab # [ 7.355792] logrotate[579]: considering log /var/log/nginx/*.log841gitlab # [ 7.357341] logrotate[579]: log /var/log/nginx/*.log does not exist -- skipping842gitlab # [ 7.358521] logrotate[579]: Creating new state843gitlab # [ 7.359257] logrotate[579]: not running postrotate script, since no logs were rotated844gitlab # [ 7.364411] logrotate[579]: switching euid from 60 to 0 and egid from 60 to 0 (pid 579)845gitlab # [ 7.365868] systemd[1]: Started Reset console on configuration changes.846gitlab # [ 7.367245] systemd[1]: Starting resolvconf update...847gitlab # [ 7.368489] systemd[1]: Finished Virtual Console Setup.848gitlab # [ 7.369463] systemd[1]: Finished Logrotate configuration check.849gitlab # [ 7.370730] nsncd[582]: Oct 02 14:09:25.261 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"850gitlab # [ 7.375539] systemd[1]: Finished Post-Boot Actions.851gitlab # [ 7.377238] systemd[1]: Started Name Service Cache Daemon (nsncd).852gitlab # [ 7.378450] systemd[1]: Reached target Host and Network Name Lookups.853gitlab # [ 7.379734] systemd[1]: Reached target User and Group Name Lookups.854gitlab # [ 7.381347] systemd[1]: Starting User Login Management...855gitlab # [ 7.382298] systemd[1]: Finished Import lastlog data into lastlog2 database.856gitlab # [ 7.427248] systemd-logind[616]: New seat seat0.857gitlab # [ 7.431180] systemd-logind[616]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)858gitlab # [ 7.432793] systemd-logind[616]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)859gitlab # [ 7.434201] systemd-logind[616]: Watching system buttons on /dev/input/event0 (gpio-keys)860gitlab # [ 7.435467] systemd[1]: Started User Login Management.861gitlab # [ 7.439689] systemd[1]: Stopped target Host and Network Name Lookups.862gitlab # [ 7.444102] systemd[1]: Stopping Host and Network Name Lookups...863gitlab # [ 7.445245] systemd[1]: Stopped target User and Group Name Lookups.864gitlab # [ 7.446241] systemd[1]: Stopping User and Group Name Lookups...865gitlab # [ 7.447215] systemd[1]: Starting linger-users.service...866gitlab # [ 7.448220] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...867gitlab # [ 7.450055] systemd[1]: nscd.service: Deactivated successfully.868gitlab # [ 7.461475] systemd[1]: Stopped Name Service Cache Daemon (nsncd).869gitlab # [ 7.462569] systemd[1]: nscd.service: Consumed 24ms CPU time over 149ms wall clock time, 7.7M memory peak, 5.8M read from disk.870gitlab # [ 7.472293] systemd[1]: Starting Name Service Cache Daemon (nsncd)...871gitlab # [ 7.477516] systemd[1]: linger-users.service: Deactivated successfully.872gitlab # [ 7.480243] systemd[1]: Finished linger-users.service.873gitlab # [ 7.483782] systemd[1]: Finished resolvconf update.874gitlab # [ 7.484764] systemd[1]: Reached target Preparation for Network.875gitlab # [ 7.487864] systemd[1]: Starting DHCP Client...876gitlab # [ 7.491049] nsncd[681]: Oct 02 14:09:25.397 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"877gitlab # [ 7.495484] systemd[1]: Starting Address configuration of eth1...878gitlab # [ 7.498111] systemd[1]: Starting Extra networking commands....879gitlab # [ 7.501286] systemd[1]: Started Name Service Cache Daemon (nsncd).880gitlab # [ 7.502314] systemd[1]: Reached target Host and Network Name Lookups.881gitlab # [ 7.503954] systemd[1]: Reached target User and Group Name Lookups.882gitlab # [ 7.541892] network-addresses-eth1-start[693]: adding address 192.168.1.1/24... done883gitlab # [ 7.548643] network-addresses-eth1-start[693]: adding address 2001:db8:1::1/64... done884gitlab # [ 7.557411] systemd[1]: Finished Address configuration of eth1.885gitlab # [ 7.584498] dhcpcd[705]: dhcpcd-10.3.2 starting886gitlab # [ 7.586850] dhcpcd[758]: dev: loaded udev887gitlab # [ 7.595670] systemd[1]: Finished Extra networking commands..888gitlab # [ 7.596854] systemd[1]: Reached target Network.889gitlab # [ 7.602294] 8021q: 802.1Q VLAN Support v1.8890gitlab # [ 7.602735] 8021q: adding VLAN 0 to HW filter on device eth1891gitlab # [ 7.600112] systemd[1]: Starting gitlab-workhorse.service...892gitlab # [ 7.606228] systemd[1]: Starting Nginx Web Server...893gitlab # [ 7.609985] systemd[1]: Starting PostgreSQL Server...894gitlab # [ 7.619104] systemd[1]: Starting Redis Server - redis-gitlab...895gitlab # [ 7.623113] systemd[1]: Starting Permit User Sessions...896gitlab # [ 7.630480] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.897gitlab # [ 7.646303] cfg80211: Loading compiled-in X.509 certificates for regulatory database898gitlab # [ 7.651336] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'899gitlab # [ 7.651842] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'900gitlab # [ 7.653243] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2901gitlab # [ 7.653583] cfg80211: failed to load regulatory.db902gitlab # [ 7.668213] systemd[1]: Finished Permit User Sessions.903gitlab # [ 7.677396] systemd[1]: Started Getty on tty1.904gitlab # [ 7.679289] systemd[1]: Reached target Login Prompts.905gitlab # [ 7.692505] 8021q: adding VLAN 0 to HW filter on device eth0906gitlab # [ 7.691582] dhcpcd[758]: eth0: waiting for carrier907gitlab # [ 7.692776] dhcpcd[758]: eth0: carrier acquired908gitlab # [ 7.700532] dhcpcd[758]: DUID 00:01:00:01:32:52:77:95:52:54:00:12:34:56909gitlab # [ 7.701713] dhcpcd[758]: eth0: IAID 00:12:34:56910gitlab # [ 7.704802] dhcpcd[758]: eth0: adding address fe80::5054:ff:fe12:3456911gitlab # [ 7.798856] nginx-pre-start[799]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok912gitlab # [ 7.800944] nginx-pre-start[799]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful913gitlab # [ 7.812961] systemd[1]: Started Nginx Web Server.914gitlab # [ 7.830775] redis-server[798]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.915gitlab # [ 7.833697] redis-server[798]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo916gitlab # [ 7.835118] redis-server[798]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=798, just started917gitlab # [ 7.836990] systemd[1]: Started Redis Server - redis-gitlab.918gitlab # [ 7.838431] postgresql-pre-start[801]: The files belonging to this database system will be owned by user "postgres".919gitlab # [ 7.840431] postgresql-pre-start[801]: This user must also own the server process.920gitlab # [ 7.841913] redis-server[798]: Configuration loaded921gitlab # [ 7.842931] postgresql-pre-start[801]: The database cluster will be initialized with locale "en_US.UTF-8".922gitlab # [ 7.844615] postgresql-pre-start[801]: The default database encoding has accordingly been set to "UTF8".923gitlab # [ 7.846086] postgresql-pre-start[801]: The default text search configuration will be set to "english".924gitlab # [ 7.847418] postgresql-pre-start[801]: Data page checksums are enabled.925gitlab # [ 7.848517] postgresql-pre-start[801]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok926gitlab # [ 7.850654] redis[798]: monotonic clock: ARM CNTVCT @ 1000 ticks/us927gitlab # [ 7.852131] postgresql-pre-start[801]: creating subdirectories ... ok928gitlab # [ 7.854324] postgresql-pre-start[801]: selecting dynamic shared memory implementation ... posix929gitlab # [ 7.855671] redis[798]: Running mode=standalone, port=0.930gitlab # [ 7.856899] redis[798]: Server initialized931gitlab # [ 7.857793] redis[798]: Ready to accept connections unix932gitlab # [ 7.858803] redis[798]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.933gitlab # [ 7.891264] postgresql-pre-start[801]: selecting default "max_connections" ... 100934gitlab # [ 7.944973] postgresql-pre-start[801]: selecting default "shared_buffers" ... 128MB935gitlab # [ 8.337314] systemd[1]: Started gitlab-workhorse.service.936gitlab # [ 8.407800] postgresql-pre-start[801]: selecting default time zone ... UTC937gitlab # [ 8.411349] postgresql-pre-start[801]: creating configuration files ... ok938gitlab # [ 8.613793] gitlab-workhorse[823]: time="2026-10-02T14:09:26Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4939gitlab # [ 8.616712] gitlab-workhorse[823]: time="2026-10-02T14:09:26Z" level=info msg="Using redis/go-redis"940gitlab # [ 8.621932] gitlab-workhorse[823]: time="2026-10-02T14:09:26Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix941gitlab # [ 8.660186] postgresql-pre-start[801]: running bootstrap script ... ok942gitlab # [ 8.804833] dhcpcd[758]: eth0: soliciting a DHCP lease943gitlab # [ 8.820701] dhcpcd[758]: eth0: offered 10.0.2.15 from 10.0.2.2944gitlab # [ 8.840383] dhcpcd[758]: eth0: probing address 10.0.2.15/24945gitlab # [ 9.333550] postgresql-pre-start[801]: performing post-bootstrap initialization ... ok946gitlab # [ 9.547942] postgresql-pre-start[801]: syncing data to disk ... ok947gitlab # [ 9.549436] postgresql-pre-start[801]: initdb: warning: enabling "trust" authentication for local connections948gitlab # [ 9.551317] postgresql-pre-start[801]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.949gitlab # [ 9.554478] postgresql-pre-start[801]: Success. You can now start the database server using:950gitlab # [ 9.556299] postgresql-pre-start[801]: pg_ctl -D /var/lib/postgresql/18 -l logfile start951gitlab # [ 9.693401] postgres[859]: [859] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit952gitlab # [ 9.696558] postgres[859]: [859] LOG: listening on IPv6 address "::1", port 5432953gitlab # [ 9.698511] postgres[859]: [859] LOG: listening on IPv4 address "127.0.0.1", port 5432954gitlab # [ 9.701480] postgres[859]: [859] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"955gitlab # [ 9.709452] postgres[868]: [868] LOG: database system was shut down at 2026-10-02 14:09:27 GMT956gitlab # [ 9.720589] postgres[859]: [859] LOG: database system is ready to accept connections957gitlab # [ 9.723953] systemd[1]: Started PostgreSQL Server.958gitlab # [ 9.728869] systemd[1]: Starting PostgreSQL Setup Scripts...959gitlab # [ 9.872593] dhcpcd[758]: eth0: soliciting an IPv6 router960gitlab # [ 9.874051] dhcpcd[758]: eth0: Router Advertisement from fe80::2961gitlab # [ 9.875499] dhcpcd[758]: eth0: adding address fec0::5054:ff:fe12:3456/64962gitlab # [ 9.877175] dhcpcd[758]: eth0: adding route to fec0::/64963gitlab # [ 9.878457] dhcpcd[758]: eth0: adding default route via fe80::2964gitlab # [ 9.975736] postgresql-setup-start[879]: CREATE DATABASE965gitlab # [ 10.017107] postgresql-setup-start[884]: CREATE ROLE966gitlab # [ 10.040788] postgresql-setup-start[886]: ALTER DATABASE967gitlab # [ 10.082570] postgresql-setup-start[891]: CREATE ROLE968gitlab # [ 10.086400] systemd[1]: Finished PostgreSQL Setup Scripts.969gitlab # [ 10.088096] systemd[1]: Reached target PostgreSQL.970gitlab # [ 10.092644] systemd[1]: Starting gitlab-postgresql.service...971gitlab # [ 10.246178] gitlab-postgresql-start[903]: CREATE DATABASE972gitlab # [ 10.279253] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[898]: mkdir -p /var/gitlab/state/home/.ssh: OK973gitlab # [ 10.281560] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[898]: chmod 700 /var/gitlab/state/home/.ssh: OK974gitlab # [ 10.294847] gitlab-postgresql-start[908]: CREATE EXTENSION975gitlab # [ 10.349840] systemd[1]: Finished gitlab-config.service.976gitlab # [ 10.354228] systemd[1]: Started gitaly.service.977gitlab # [ 10.359634] systemd[1]: Starting gitlab-db-config.service...978gitlab # [ 10.388394] gitlab-postgresql-start[914]: CREATE EXTENSION979gitlab # [ 10.389997] systemd[1]: Finished gitlab-postgresql.service.980gitlab # [ 11.026266] gitaly[924]: time="2026-10-02T14:09:28.932Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=924981gitlab # [ 11.026506] gitaly[924]: time="2026-10-02T14:09:28.932Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=924982gitlab # [ 11.026568] gitaly[924]: time="2026-10-02T14:09:28.932Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=924983gitlab # [ 11.026619] gitaly[924]: time="2026-10-02T14:09:28.932Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=924984gitlab # [ 11.026669] gitaly[924]: time="2026-10-02T14:09:28.932Z" level=info msg="Starting Gitaly" pid=924 version=985gitlab # [ 11.250118] gitaly[924]: time="2026-10-02T14:09:29.157Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=223 pid=924986gitlab # [ 11.252680] gitaly[924]: time="2026-10-02T14:09:29.157Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=924987gitlab # [ 11.254581] gitaly[924]: time="2026-10-02T14:09:29.158Z" level=info msg="finished initializing command factory" duration_ms=0 pid=924988gitlab # [ 11.256288] gitaly[924]: time="2026-10-02T14:09:29.158Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-924/git-exec-4152917027.d/git pid=924989gitlab # [ 11.259278] gitaly[924]: time="2026-10-02T14:09:29.166Z" level=info msg="finished detecting git version" duration_ms=8 pid=924990gitlab # [ 11.261658] gitaly[924]: time="2026-10-02T14:09:29.166Z" level=info msg="using Git version" pid=924 version=2.53.g991gitlab # [ 11.263736] gitaly[924]: time="2026-10-02T14:09:29.166Z" level=info msg="clearing disk cache object folder" pid=924 storage=default992gitlab # [ 11.265912] gitaly[924]: time="2026-10-02T14:09:29.167Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1405280717 pid=924 storage=default993gitlab # [ 11.269213] gitaly[924]: time="2026-10-02T14:09:29.167Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1405280717 pid=924 storage=default994gitlab # [ 11.272577] gitaly[924]: time="2026-10-02T14:09:29.167Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=924995gitlab # [ 11.276198] gitaly[924]: time="2026-10-02T14:09:29.167Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=924996gitlab # [ 11.279629] gitaly[924]: time="2026-10-02T14:09:29.168Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1405280717 clear_duration_ms=0 pid=924 storage=default997gitlab # [ 11.283691] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=924998gitlab # [ 11.286615] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=924 schema=unix999gitlab # [ 11.289078] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=info msg="counting repositories" pid=9241000gitlab # [ 11.290756] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=info msg="listening at address" address=/run/gitaly/gitaly-924/sock.d/intern pid=924 schema=unix1001gitlab # [ 11.293346] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=info msg="finished tempdir cleaner walk" pid=924 storage=default time_ms=01002gitlab # [ 11.295408] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=info msg="starting to count repositories" pid=924 storage_path=/var/gitlab/state/repositories1003gitlab # [ 11.297943] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=924 storage_path=/var/gitlab/state/repositories1004gitlab # [ 11.300612] gitaly[924]: time="2026-10-02T14:09:29.184Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9241005gitlab # [ 11.302655] gitaly[924]: time="2026-10-02T14:09:29.186Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9241006gitlab # [ 11.304772] gitaly[924]: time="2026-10-02T14:09:29.186Z" level=info msg="Started Gitaly" duration_ms=253 pid=9241007gitlab # [ 11.306459] gitaly[924]: time="2026-10-02T14:09:29.186Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9241008gitlab # [ 11.308495] gitaly[924]: time="2026-10-02T14:09:29.186Z" level=info msg="maintenance: daily scheduled" pid=924 scheduled="2026-10-03 12:00:00 +0000 UTC"1009gitlab # [ 11.533515] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[928]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1010gitlab # [ 11.538129] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[928]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1011gitlab # [ 11.552057] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[928]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1012gitlab # [ 14.274520] dhcpcd[758]: eth0: leased 10.0.2.15 for 86400 seconds1013gitlab # [ 14.276179] dhcpcd[758]: eth0: adding route to 10.0.2.0/241014gitlab # [ 14.277458] dhcpcd[758]: eth0: adding default route via 10.0.2.21015gitlab # [ 14.381381] systemd[1]: Started DHCP Client.1016gitlab # [ 14.383594] systemd[1]: Reached target Network is Online.1017gitlab # [ 14.757184] gitaly[924]: time="2026-10-02T14:09:32.663Z" level=info msg="License database preloaded" duration_ms=3730 pid=9241018gitlab # [ 57.408162] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[928]: W, [2026-10-02T14:10:15.314774 #928] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1019gitlab # [ 58.688500] gitlab-workhorse[823]: time="2026-10-02T14:10:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1020gitlab # [ 60.559457] postgres[1459]: [1459] ERROR: relation "feature_gates" does not exist at character 5231021gitlab # [ 60.559729] postgres[1459]: [1459] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1022gitlab # [ 60.559798] postgres[1459]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1023gitlab # [ 60.559852] postgres[1459]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1024gitlab # [ 60.559911] postgres[1459]: attidentity AS identity,1025gitlab # [ 60.559970] postgres[1459]: attgenerated as attgenerated1026gitlab # [ 60.560096] postgres[1459]: FROM pg_attribute a1027gitlab # [ 60.560159] postgres[1459]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1028gitlab # [ 60.560229] postgres[1459]: LEFT JOIN pg_type t ON a.atttypid = t.oid1029gitlab # [ 60.560288] postgres[1459]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1030gitlab # [ 60.560348] postgres[1459]: WHERE a.attrelid = '"feature_gates"'::regclass1031gitlab # [ 60.560408] postgres[1459]: AND a.attnum > 0 AND NOT a.attisdropped1032gitlab # [ 60.560469] postgres[1459]: ORDER BY a.attnum1033gitlab # [ 60.560528] postgres[1459]:1034gitlab # [ 66.188377] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1510]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1035gitlab # [ 66.190883] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1510]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1036gitlab # [ 68.698553] gitlab-workhorse[823]: time="2026-10-02T14:10:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1037gitlab # [ 78.708268] gitlab-workhorse[823]: time="2026-10-02T14:10:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 88.717891] gitlab-workhorse[823]: time="2026-10-02T14:10:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1039gitlab # [ 89.804872] postgres[1622]: [1622] ERROR: relation "table_without_models" does not exist1040gitlab # [ 89.805140] postgres[1622]: [1622] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1041gitlab # [ 98.727649] gitlab-workhorse[823]: time="2026-10-02T14:10:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1042gitlab # [ 108.737361] gitlab-workhorse[823]: time="2026-10-02T14:11:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1043gitlab # [ 110.289887] systemd[1]: Finished gitlab-db-config.service.1044gitlab # [ 110.293270] systemd[1]: Started gitlab-sidekiq.service.1045gitlab # [ 110.296333] systemd[1]: Starting gitlab.service...1046gitlab # [ 110.849156] bundle[1898]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1047gitlab # [ 110.852751] bundle[1898]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1048gitlab # [ 110.855989] bundle[1898]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1049gitlab # [ 116.021830] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.893Z","pid":1898,"message":"* Enabling systemd notification integration"}1050gitlab # [ 116.022145] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"Puma starting in cluster mode..."}1051gitlab # [ 116.022208] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1052gitlab # [ 116.022261] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1053gitlab # [ 116.022320] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"* Min threads: 0"}1054gitlab # [ 116.022371] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"* Max threads: 4"}1055gitlab # [ 116.022429] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"* Environment: production"}1056gitlab # [ 116.022480] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"* Master PID: 1898"}1057gitlab # [ 116.022537] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.928Z","pid":1898,"message":"* Workers: 2"}1058gitlab # [ 116.022613] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.929Z","pid":1898,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1059gitlab # [ 116.022665] bundle[1898]: {"timestamp":"2026-10-02T14:11:13.929Z","pid":1898,"message":"* Preloading application"}1060gitlab # [ 116.126303] sidekiq-cluster[1897]: {"severity":"INFO","time":"2026-10-02T14:11:13.999Z","message":"Starting cluster with 1 processes"}1061gitlab # [ 118.749364] gitlab-workhorse[823]: time="2026-10-02T14:11:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1062gitlab # [ 128.750489] gitlab-workhorse[823]: time="2026-10-02T14:11:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1063gitlab # [ 138.751625] gitlab-workhorse[823]: time="2026-10-02T14:11:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1064gitlab # [ 148.752562] gitlab-workhorse[823]: time="2026-10-02T14:11:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1065gitlab # [ 158.415989] bundle[1898]: W, [2026-10-02T14:11:56.322574 #1898] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1066gitlab # [ 158.761599] gitlab-workhorse[823]: time="2026-10-02T14:11:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1067gitlab # [ 164.540482] sidekiq-cluster[1955]: 2026-10-02T14:12:02.446Z pid=1955 tid=wln INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1955, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430350336}1068gitlab # [ 164.892752] sidekiq-cluster[1955]: W, [2026-10-02T14:12:02.799916 #1955] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1069gitlab # [ 164.963280] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:02.870Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1070gitlab # [ 168.769508] gitlab-workhorse[823]: time="2026-10-02T14:12:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1071gitlab # [ 169.158658] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.065Z","message":"GitLab reliable fetch activated!"}1072gitlab # [ 169.161620] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.065Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1073gitlab # [ 169.194330] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.101Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1074gitlab # [ 169.198120] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.104Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1075gitlab # [ 169.201914] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.107Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1076gitlab # [ 169.205506] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.109Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1077gitlab # [ 169.209462] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.112Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1078gitlab # [ 169.214281] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.115Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1079gitlab # [ 169.220276] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.117Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1080gitlab # [ 169.222584] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.119Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1081gitlab # [ 169.224922] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.121Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1082gitlab # [ 169.227176] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.127Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1083gitlab # [ 169.229812] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.131Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1084gitlab # [ 169.232222] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.135Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1085gitlab # [ 169.235349] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.139Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1086gitlab # [ 169.237881] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.142Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1087gitlab # [ 169.240264] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.146Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1088gitlab # [ 169.242894] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.150Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1089gitlab # [ 169.254658] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.161Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1090gitlab # [ 169.260833] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.168Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1091gitlab # [ 169.264914] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.172Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1092gitlab # [ 169.268621] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.176Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1093gitlab # [ 169.272459] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.179Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1094gitlab # [ 169.275274] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.182Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1095gitlab # [ 169.278094] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.185Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1096gitlab # [ 169.281389] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.188Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1097gitlab # [ 169.284462] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.191Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1098gitlab # [ 169.288070] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.195Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1099gitlab # [ 169.291383] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.198Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1100gitlab # [ 169.294579] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.201Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1101gitlab # [ 169.297816] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.205Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1102gitlab # [ 169.300798] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.208Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1103gitlab # [ 169.303698] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.211Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1104gitlab # [ 169.306699] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.214Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1105gitlab # [ 169.309484] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.216Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1106gitlab # [ 169.312124] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.219Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1107gitlab # [ 169.314709] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.222Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1108gitlab # [ 169.317132] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.224Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1109gitlab # [ 169.319459] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.226Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1110gitlab # [ 169.321858] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.229Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1111gitlab # [ 169.324140] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.231Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1112gitlab # [ 169.326350] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.233Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1113gitlab # [ 169.328747] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.235Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1114gitlab # [ 169.330939] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.236Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1115gitlab # [ 169.333277] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.238Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1116gitlab # [ 169.335401] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.241Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1117gitlab # [ 169.337797] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.243Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1118gitlab # [ 169.340169] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.245Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1119gitlab # [ 169.342398] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.247Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1120gitlab # [ 169.344655] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.249Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1121gitlab # [ 169.346801] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.251Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1122gitlab # [ 169.349051] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.253Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1123gitlab # [ 169.351217] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.255Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1124gitlab # [ 169.353498] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.258Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1125gitlab # [ 169.361966] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.269Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1126gitlab # [ 169.364539] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.271Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1127gitlab # [ 169.366863] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.274Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1128gitlab # [ 169.369176] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.276Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1129gitlab # [ 169.371429] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.278Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1130gitlab # [ 169.373967] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.280Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1131gitlab # [ 169.376309] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.282Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1132gitlab # [ 169.378629] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.284Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1133gitlab # [ 169.380776] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.286Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1134gitlab # [ 169.382861] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.288Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1135gitlab # [ 169.385150] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.290Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1136gitlab # [ 169.387421] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.292Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1137gitlab # [ 169.389855] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.294Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1138gitlab # [ 169.392084] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.297Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1139gitlab # [ 169.394221] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.298Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1140gitlab # [ 169.396511] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.300Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1141gitlab # [ 169.398871] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.302Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1142gitlab # [ 169.401331] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.303Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1143gitlab # [ 169.403637] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.305Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1144gitlab # [ 169.406096] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.307Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1145gitlab # [ 169.408424] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.309Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1146gitlab # [ 169.410829] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.310Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1147gitlab # [ 169.413305] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.312Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1148gitlab # [ 169.415774] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.314Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1149gitlab # [ 169.418230] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.316Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1150gitlab # [ 169.420846] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.317Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1151gitlab # [ 169.423106] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.319Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1152gitlab # [ 169.425439] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.321Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1153gitlab # [ 169.427853] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.323Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1154gitlab # [ 169.430561] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.325Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1155gitlab # [ 169.433025] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.328Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1156gitlab # [ 169.435363] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.329Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1157gitlab # [ 169.437946] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.331Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1158gitlab # [ 169.441211] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.333Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1159gitlab # [ 169.443647] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.336Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1160gitlab # [ 169.446204] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.338Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1161gitlab # [ 169.448659] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.340Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1162gitlab # [ 169.451027] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.341Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1163gitlab # [ 169.453527] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.343Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1164gitlab # [ 169.455871] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.345Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1165gitlab # [ 169.458428] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.347Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1166gitlab # [ 169.461013] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.349Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1167gitlab # [ 169.463183] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.351Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1168gitlab # [ 169.465790] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.354Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1169gitlab # [ 169.468190] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.356Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1170gitlab # [ 169.470554] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.358Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1171gitlab # [ 169.472971] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.372Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1172gitlab # [ 169.475326] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.374Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1173gitlab # [ 169.477819] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.376Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1174gitlab # [ 169.480207] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.379Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1175gitlab # [ 169.482382] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.382Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1176gitlab # [ 169.484725] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.385Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1177gitlab # [ 169.536607] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.441Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1178gitlab # [ 169.539357] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.446Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1179gitlab # [ 169.541737] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.449Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1180gitlab # [ 169.545087] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.452Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1181gitlab # [ 169.548799] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:07.456Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1182gitlab # [ 178.777458] gitlab-workhorse[823]: time="2026-10-02T14:12:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1183gitlab # [ 188.788486] gitlab-workhorse[823]: time="2026-10-02T14:12:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1184gitlab # [ 198.798316] gitlab-workhorse[823]: time="2026-10-02T14:12:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1185gitlab # [ 208.799128] gitlab-workhorse[823]: time="2026-10-02T14:12:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1186gitlab # [ 209.478312] postgres[2782]: [2782] ERROR: relation "table_without_models" does not exist1187gitlab # [ 209.480403] postgres[2782]: [2782] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1188gitlab # [ 210.783242] postgres[2793]: [2793] ERROR: relation "table_without_models" does not exist1189gitlab # [ 210.783507] postgres[2793]: [2793] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1190gitlab # [ 215.377324] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.283Z","pid":1898,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1191gitlab # [ 215.377606] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.284Z","pid":1898,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1192gitlab # [ 215.377725] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.284Z","pid":1898,"message":"! #\u003cThread:0x0000e93f78283240 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1193gitlab # [ 215.377798] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.284Z","pid":1898,"message":"! #\u003cThread:0x0000e93f7853b640 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1194gitlab # [ 215.377860] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.284Z","pid":1898,"message":"Use Ctrl-C to stop"}1195gitlab # [ 215.377934] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.284Z","pid":1898,"message":"Sending status to systemd every 1.0 sec"}1196gitlab # [ 215.986951] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.892Z","pid":1898,"message":"- Worker 0 (PID: 2866) booted in 0.57s, phase: 0"}1197gitlab # [ 216.017714] bundle[1898]: {"timestamp":"2026-10-02T14:12:53.924Z","pid":1898,"message":"- Worker 1 (PID: 2870) booted in 0.58s, phase: 0"}1198gitlab # [ 216.023098] systemd[1]: Started gitlab.service.1199gitlab # [ 216.024154] systemd[1]: Reached target Common target for all GitLab services..1200gitlab # [ 216.028887] systemd[1]: Starting setup-gitlab.service...1201gitlab # [ 216.075688] setup-gitlab-start[2905]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1202gitlab # [ 216.187254] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:54.092Z","message":"Booted Rails 7.2.3.1 application in production environment"}1203gitlab # [ 216.190841] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:54.093Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1204gitlab # [ 216.194444] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:54.093Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1205gitlab # [ 216.197549] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:54.093Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1206gitlab # [ 216.200935] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:54.096Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1207gitlab # [ 219.227241] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:57.133Z","message":"Cleaning working queues"}1208gitlab # [ 219.230141] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:57.135Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"8da45f23d41df0ab2ed0cb8f","created_at":"2026-10-02T14:11:07.694Z","trace_propagation_headers":{"sentry-trace":"f5231d7900284381b7e56ca41af2430e-d08afcab55d14416","baggage":"sentry-trace_id=f5231d7900284381b7e56ca41af2430e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"0a3d8f8e06e154945a8320fc39f81024","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:11:07.706Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-8da45f23d41df0ab2ed0cb8f: start","job_status":"start","queue_duration_s":109.428483,"scheduling_latency_s":109.428542}1209gitlab # [ 219.458437] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:12:57.365Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"8da45f23d41df0ab2ed0cb8f","created_at":"2026-10-02T14:11:07.694Z","trace_propagation_headers":{"sentry-trace":"f5231d7900284381b7e56ca41af2430e-d08afcab55d14416","baggage":"sentry-trace_id=f5231d7900284381b7e56ca41af2430e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"0a3d8f8e06e154945a8320fc39f81024","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:11:07.706Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-8da45f23d41df0ab2ed0cb8f: done: 0.229885 sec","job_status":"done","queue_duration_s":109.428483,"scheduling_latency_s":109.428542,"redis_calls":24,"redis_duration_s":0.007056,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003277,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002187,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001592,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1357431,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.229885,"completed_at":"2026-10-02T14:12:57.364Z","load_balancing_strategy":"primary","db_duration_s":0.072138,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1210gitlab # [ 220.279710] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:12:58 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 41711211gitlab # [ 220.284203] setup-gitlab-start[2905]: + TOKEN=nixbot-test-token-of-20-chars1212gitlab # [ 220.286357] setup-gitlab-start[2905]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1213gitlab # [ 220.296272] runuser[2942]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1214gitlab # [ 220.879719] setup-gitlab-start[2943]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1215gitlab # [ 220.879996] setup-gitlab-start[2943]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1216gitlab # [ 220.882339] setup-gitlab-start[2943]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1217gitlab # [ 221.761246] setup-gitlab-start[2943]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1218gitlab # [ 221.761683] setup-gitlab-start[2943]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1219gitlab # [ 221.768675] setup-gitlab-start[2943]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1220gitlab # [ 229.362828] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b2a705ece0a8e2d2e808bd8d","created_at":"2026-10-02T14:13:07.198Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f10c874bc1196ef1173575c2b02e533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.265Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b2a705ece0a8e2d2e808bd8d: start","job_status":"start","queue_duration_s":0.003822,"scheduling_latency_s":0.003876}1221gitlab # [ 229.743249] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b2a705ece0a8e2d2e808bd8d","created_at":"2026-10-02T14:13:07.198Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f10c874bc1196ef1173575c2b02e533","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.265Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b2a705ece0a8e2d2e808bd8d: done: 0.380578 sec","job_status":"done","queue_duration_s":0.003822,"scheduling_latency_s":0.003876,"redis_calls":12,"redis_duration_s":0.02357,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003051,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009726,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0019483,"exclusive_lock_hold_duration_s":0.18787272,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.380578,"completed_at":"2026-10-02T14:13:07.649Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.065596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1222gitlab # [ 229.810878] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.659Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0f98a803022205d19260d71e","created_at":"2026-10-02T14:13:07.287Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0727aef7a631aed9237313a0b473ef00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.300Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0f98a803022205d19260d71e: start","job_status":"start","queue_duration_s":0.358716,"scheduling_latency_s":0.358773}1223gitlab # [ 229.828939] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.733Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1224gitlab # [ 229.855848] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"0f98a803022205d19260d71e","created_at":"2026-10-02T14:13:07.287Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0727aef7a631aed9237313a0b473ef00","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.300Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-0f98a803022205d19260d71e: done: 0.103359 sec","job_status":"done","queue_duration_s":0.358716,"scheduling_latency_s":0.358773,"redis_calls":10,"redis_duration_s":0.028288,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.015127,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001995,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00289362,"duration_s":0.103359,"completed_at":"2026-10-02T14:13:07.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1225gitlab # [ 229.911825] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"40c0baadf538713d864f8bb8","created_at":"2026-10-02T14:13:07.318Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.336Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-40c0baadf538713d864f8bb8: start","job_status":"start","queue_duration_s":0.429193,"scheduling_latency_s":0.429245}1226gitlab # [ 230.039894] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.946Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"40c0baadf538713d864f8bb8","created_at":"2026-10-02T14:13:07.318Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.336Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-40c0baadf538713d864f8bb8: done: 0.180694 sec","job_status":"done","queue_duration_s":0.429193,"scheduling_latency_s":0.429245,"redis_calls":19,"redis_duration_s":0.016963,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004138,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.003005,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.007273,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002547,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00147386,"duration_s":0.180694,"completed_at":"2026-10-02T14:13:07.946Z","load_balancing_strategy":"primary","db_duration_s":0.019473,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1227gitlab # [ 230.100406] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:07.953Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bc1dceb0acbc6eb103c810da","created_at":"2026-10-02T14:13:07.345Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25d6d5e96a065e443ec2fe0a9d02cfde","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.357Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bc1dceb0acbc6eb103c810da: start","job_status":"start","queue_duration_s":0.596305,"scheduling_latency_s":0.596352}1228gitlab # [ 230.215875] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.122Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"bc1dceb0acbc6eb103c810da","created_at":"2026-10-02T14:13:07.345Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25d6d5e96a065e443ec2fe0a9d02cfde","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.357Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-bc1dceb0acbc6eb103c810da: done: 0.168894 sec","job_status":"done","queue_duration_s":0.596305,"scheduling_latency_s":0.596352,"redis_calls":14,"redis_duration_s":0.014342,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004802,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00205,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.039,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00122234,"exclusive_lock_hold_duration_s":0.09789479999999999,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.168894,"completed_at":"2026-10-02T14:13:08.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.076408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1229gitlab # [ 230.281719] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.124Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"dc462420271dd6691fe36af0","created_at":"2026-10-02T14:13:07.378Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16e4849e3b4618b4cf9a966434b25566","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.392Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-dc462420271dd6691fe36af0: start","job_status":"start","queue_duration_s":0.731968,"scheduling_latency_s":0.732009}1230gitlab # [ 230.296493] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"dc462420271dd6691fe36af0","created_at":"2026-10-02T14:13:07.378Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16e4849e3b4618b4cf9a966434b25566","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.392Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-dc462420271dd6691fe36af0: done: 0.048896 sec","job_status":"done","queue_duration_s":0.731968,"scheduling_latency_s":0.732009,"redis_calls":12,"redis_duration_s":0.002801,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000478,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001292,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00077796,"exclusive_lock_hold_duration_s":0.017097519999999998,"duration_s":0.048896,"completed_at":"2026-10-02T14:13:08.173Z","load_balancing_strategy":"primary","db_duration_s":0.005564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1231gitlab # [ 230.342023] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"6798f91605a3d19526844aa9","created_at":"2026-10-02T14:13:07.416Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0663c07b402342fe364eb49362b8c47b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.437Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-6798f91605a3d19526844aa9: start","job_status":"start","queue_duration_s":0.738431,"scheduling_latency_s":0.73847}1232gitlab # [ 230.358029] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.194Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1233gitlab # [ 230.361785] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.205Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"6798f91605a3d19526844aa9","created_at":"2026-10-02T14:13:07.416Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0663c07b402342fe364eb49362b8c47b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.437Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-6798f91605a3d19526844aa9: done: 0.029513 sec","job_status":"done","queue_duration_s":0.738431,"scheduling_latency_s":0.73847,"redis_calls":10,"redis_duration_s":0.002405,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000467,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039114,"duration_s":0.029513,"completed_at":"2026-10-02T14:13:08.205Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1234gitlab # [ 230.409051] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.207Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"26734d8dd3b72c27876db0d9","created_at":"2026-10-02T14:13:07.477Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81bc3e11547ffad9452ffb3172c1962e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.484Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-26734d8dd3b72c27876db0d9: start","job_status":"start","queue_duration_s":0.722529,"scheduling_latency_s":0.722567}1235gitlab # [ 230.425083] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.227Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1236gitlab # [ 230.429242] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.238Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"26734d8dd3b72c27876db0d9","created_at":"2026-10-02T14:13:07.477Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"81bc3e11547ffad9452ffb3172c1962e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.484Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-26734d8dd3b72c27876db0d9: done: 0.031466 sec","job_status":"done","queue_duration_s":0.722529,"scheduling_latency_s":0.722567,"redis_calls":10,"redis_duration_s":0.002515,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000554,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042594,"duration_s":0.031466,"completed_at":"2026-10-02T14:13:08.238Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000733,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1237gitlab # [ 230.475769] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b5121d61344f3a6e71679a5e","created_at":"2026-10-02T14:13:07.493Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.500Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b5121d61344f3a6e71679a5e: start","job_status":"start","queue_duration_s":0.740441,"scheduling_latency_s":0.740479}1238gitlab # [ 230.519844] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b5121d61344f3a6e71679a5e","created_at":"2026-10-02T14:13:07.493Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.500Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b5121d61344f3a6e71679a5e: done: 0.185825 sec","job_status":"done","queue_duration_s":0.740441,"scheduling_latency_s":0.740479,"redis_calls":71,"redis_duration_s":0.019226,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002583,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001098,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.01472,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000825,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.165265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041712,"duration_s":0.185825,"completed_at":"2026-10-02T14:13:08.426Z","load_balancing_strategy":"primary","db_duration_s":0.003084,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1239gitlab # [ 230.567520] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"623b88066371d277d8b32a0a","created_at":"2026-10-02T14:13:07.504Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2a74ecbeb1f27ce8e304333cf46e724","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.513Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-623b88066371d277d8b32a0a: start","job_status":"start","queue_duration_s":0.91494,"scheduling_latency_s":0.914984}1240gitlab # [ 230.581950] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"623b88066371d277d8b32a0a","created_at":"2026-10-02T14:13:07.504Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e2a74ecbeb1f27ce8e304333cf46e724","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.513Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-623b88066371d277d8b32a0a: done: 0.039539 sec","job_status":"done","queue_duration_s":0.91494,"scheduling_latency_s":0.914984,"redis_calls":12,"redis_duration_s":0.002994,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00051,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00112,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006354,"exclusive_lock_hold_duration_s":0.010009019999999999,"duration_s":0.039539,"completed_at":"2026-10-02T14:13:08.468Z","load_balancing_strategy":"primary","db_duration_s":0.005272,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1241gitlab # [ 230.625605] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.469Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d8b3dcc1c638486ee89770bb","created_at":"2026-10-02T14:13:07.522Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fc8ab5080fd251c14516f86b1fafa95","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.531Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d8b3dcc1c638486ee89770bb: start","job_status":"start","queue_duration_s":0.938756,"scheduling_latency_s":0.93878}1242gitlab # [ 230.641162] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"d8b3dcc1c638486ee89770bb","created_at":"2026-10-02T14:13:07.522Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5fc8ab5080fd251c14516f86b1fafa95","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.531Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-d8b3dcc1c638486ee89770bb: done: 0.041467 sec","job_status":"done","queue_duration_s":0.938756,"scheduling_latency_s":0.93878,"redis_calls":10,"redis_duration_s":0.0021579999999999998,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004189,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.041467,"completed_at":"2026-10-02T14:13:08.511Z","load_balancing_strategy":"primary","db_duration_s":0.015489,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 230.687538] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"28ed97f7a1f2fc929e93e0e5","created_at":"2026-10-02T14:13:07.549Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7591e3c32ba177d52dc6370e3310f10a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.560Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-28ed97f7a1f2fc929e93e0e5: start","job_status":"start","queue_duration_s":0.952119,"scheduling_latency_s":0.952145}1244gitlab # [ 230.702941] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.537Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"28ed97f7a1f2fc929e93e0e5","created_at":"2026-10-02T14:13:07.549Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7591e3c32ba177d52dc6370e3310f10a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.560Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-28ed97f7a1f2fc929e93e0e5: done: 0.024938 sec","job_status":"done","queue_duration_s":0.952119,"scheduling_latency_s":0.952145,"redis_calls":12,"redis_duration_s":0.002311,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000805,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029458,"duration_s":0.024938,"completed_at":"2026-10-02T14:13:08.537Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1245gitlab # [ 230.747280] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"288c31b31de85d93578799b4","created_at":"2026-10-02T14:13:07.575Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"866e2f900c309a4d2025ccd7ec2aa08f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:13:07.593Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-288c31b31de85d93578799b4: start","job_status":"start","queue_duration_s":0.9462,"scheduling_latency_s":0.946224}1246gitlab # [ 230.763207] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.552Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1247gitlab # [ 230.767450] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"288c31b31de85d93578799b4","created_at":"2026-10-02T14:13:07.575Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"866e2f900c309a4d2025ccd7ec2aa08f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:13:07.593Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-288c31b31de85d93578799b4: done: 0.021764 sec","job_status":"done","queue_duration_s":0.9462,"scheduling_latency_s":0.946224,"redis_calls":10,"redis_duration_s":0.0019190000000000001,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031374,"duration_s":0.021764,"completed_at":"2026-10-02T14:13:08.561Z","load_balancing_strategy":"primary","db_duration_s":0.000573,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1248gitlab # [ 230.812119] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.562Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3e6daaf5513b63f8daded312","created_at":"2026-10-02T14:13:07.631Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff3e8c65375f7bd9a2da35391fac70f1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.657Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3e6daaf5513b63f8daded312: start","job_status":"start","queue_duration_s":0.905209,"scheduling_latency_s":0.905233}1249gitlab # [ 230.827370] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.600Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3e6daaf5513b63f8daded312","created_at":"2026-10-02T14:13:07.631Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff3e8c65375f7bd9a2da35391fac70f1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.657Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3e6daaf5513b63f8daded312: done: 0.037838 sec","job_status":"done","queue_duration_s":0.905209,"scheduling_latency_s":0.905233,"redis_calls":16,"redis_duration_s":0.003273,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001741,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001149,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000383,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022539,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029558,"duration_s":0.037838,"completed_at":"2026-10-02T14:13:08.600Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006497,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1250gitlab # [ 230.871966] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.602Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ce818a656badea7c66fb3017","created_at":"2026-10-02T14:13:07.675Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c14efffa3b1077df2ba7280e0e4d3da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.703Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ce818a656badea7c66fb3017: start","job_status":"start","queue_duration_s":0.899024,"scheduling_latency_s":0.899048}1251gitlab # [ 230.887334] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.615Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1252gitlab # [ 230.891284] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"ce818a656badea7c66fb3017","created_at":"2026-10-02T14:13:07.675Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c14efffa3b1077df2ba7280e0e4d3da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.703Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-ce818a656badea7c66fb3017: done: 0.022059 sec","job_status":"done","queue_duration_s":0.899024,"scheduling_latency_s":0.899048,"redis_calls":10,"redis_duration_s":0.002029,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000406,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031472,"duration_s":0.022059,"completed_at":"2026-10-02T14:13:08.624Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000471,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1253gitlab # [ 230.936343] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f1cd7a1b06b8b94264324617","created_at":"2026-10-02T14:13:07.730Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be49538bd75d63a0806193b7ca062f64","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.757Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f1cd7a1b06b8b94264324617: start","job_status":"start","queue_duration_s":0.868109,"scheduling_latency_s":0.868135}1254gitlab # [ 230.951838] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.656Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f1cd7a1b06b8b94264324617","created_at":"2026-10-02T14:13:07.730Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"be49538bd75d63a0806193b7ca062f64","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.757Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f1cd7a1b06b8b94264324617: done: 0.030187 sec","job_status":"done","queue_duration_s":0.868109,"scheduling_latency_s":0.868135,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00043,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032568,"duration_s":0.030187,"completed_at":"2026-10-02T14:13:08.656Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1255gitlab # [ 231.000182] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a61ceed34d9c89f03b4e6546","created_at":"2026-10-02T14:13:07.777Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de3b136ba1b48302cd124294069ca4ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:13:07.825Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a61ceed34d9c89f03b4e6546: start","job_status":"start","queue_duration_s":0.832822,"scheduling_latency_s":0.832846}1256gitlab # [ 231.014459] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.672Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1257gitlab # [ 231.018093] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.681Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a61ceed34d9c89f03b4e6546","created_at":"2026-10-02T14:13:07.777Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de3b136ba1b48302cd124294069ca4ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:13:07.825Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a61ceed34d9c89f03b4e6546: done: 0.022981 sec","job_status":"done","queue_duration_s":0.832822,"scheduling_latency_s":0.832846,"redis_calls":10,"redis_duration_s":0.002124,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001239,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033136,"duration_s":0.022981,"completed_at":"2026-10-02T14:13:08.681Z","load_balancing_strategy":"primary","db_duration_s":0.000662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 231.062457] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.683Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"87229fd3e8f05cea1ae3370b","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-87229fd3e8f05cea1ae3370b: start","job_status":"start","queue_duration_s":0.799163,"scheduling_latency_s":0.799188}1259gitlab # [ 231.075686] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.704Z","message":"No existing merge request to be cleaned up."}1260gitlab # [ 231.077576] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.721Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"87229fd3e8f05cea1ae3370b","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-87229fd3e8f05cea1ae3370b: done: 0.038185 sec","job_status":"done","queue_duration_s":0.799163,"scheduling_latency_s":0.799188,"redis_calls":13,"redis_duration_s":0.003468,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003132,"duration_s":0.038185,"completed_at":"2026-10-02T14:13:08.721Z","load_balancing_strategy":"primary","db_duration_s":0.004801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1261gitlab # [ 231.120779] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.723Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b161a6f97ed7d97171d16a55","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b161a6f97ed7d97171d16a55: start","job_status":"start","queue_duration_s":0.839143,"scheduling_latency_s":0.839167}1262gitlab # [ 231.134304] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.737Z","message":"No existing merge request to be cleaned up."}1263gitlab # [ 231.136151] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.748Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b161a6f97ed7d97171d16a55","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b161a6f97ed7d97171d16a55: done: 0.025146 sec","job_status":"done","queue_duration_s":0.839143,"scheduling_latency_s":0.839167,"redis_calls":10,"redis_duration_s":0.002802,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001822,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025146,"completed_at":"2026-10-02T14:13:08.748Z","load_balancing_strategy":"primary","db_duration_s":0.002204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1264gitlab # [ 231.177370] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.749Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f214ee4f83df202205134fa4","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f214ee4f83df202205134fa4: start","job_status":"start","queue_duration_s":0.865858,"scheduling_latency_s":0.865883}1265gitlab # [ 231.190127] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.763Z","message":"No existing merge request to be cleaned up."}1266gitlab # [ 231.191860] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.773Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f214ee4f83df202205134fa4","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f214ee4f83df202205134fa4: done: 0.024245 sec","job_status":"done","queue_duration_s":0.865858,"scheduling_latency_s":0.865883,"redis_calls":10,"redis_duration_s":0.002312,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001049,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024245,"completed_at":"2026-10-02T14:13:08.773Z","load_balancing_strategy":"primary","db_duration_s":0.001975,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1267gitlab # [ 231.232435] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.775Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b01dadadfc378ce8082f8cf5","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b01dadadfc378ce8082f8cf5: start","job_status":"start","queue_duration_s":0.891725,"scheduling_latency_s":0.891751}1268gitlab # [ 231.245231] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.789Z","message":"No existing merge request to be cleaned up."}1269gitlab # [ 231.246931] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.799Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b01dadadfc378ce8082f8cf5","created_at":"2026-10-02T14:13:07.830Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-f8fbfa0708fe4803","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"4f52aa24d2951ec0e267363700d5db39","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.883Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b01dadadfc378ce8082f8cf5: done: 0.024393 sec","job_status":"done","queue_duration_s":0.891725,"scheduling_latency_s":0.891751,"redis_calls":10,"redis_duration_s":0.002537,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001528,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024393,"completed_at":"2026-10-02T14:13:08.799Z","load_balancing_strategy":"primary","db_duration_s":0.002286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 231.287289] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:08.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4038461adc7d81dafc75ed5d","created_at":"2026-10-02T14:13:07.851Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"163a383492a677bbec5b4bacb0e59d37","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.886Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4038461adc7d81dafc75ed5d: start","job_status":"start","queue_duration_s":0.915292,"scheduling_latency_s":0.915322}1271gitlab # [ 231.302063] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.099Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"4038461adc7d81dafc75ed5d","created_at":"2026-10-02T14:13:07.851Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"163a383492a677bbec5b4bacb0e59d37","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.886Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-4038461adc7d81dafc75ed5d: done: 0.298029 sec","job_status":"done","queue_duration_s":0.915292,"scheduling_latency_s":0.915322,"redis_calls":12,"redis_duration_s":0.002617,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.045,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.239622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058888,"exclusive_lock_hold_duration_s":0.27264454000000005,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.298029,"completed_at":"2026-10-02T14:13:09.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.066797,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1272gitlab # [ 231.348683] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2a4ff939eb86b085b2f7503c","created_at":"2026-10-02T14:13:07.938Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51887796854ccc924035e58756e2b861","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:13:07.952Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2a4ff939eb86b085b2f7503c: start","job_status":"start","queue_duration_s":1.149131,"scheduling_latency_s":1.149159}1273gitlab # [ 231.363508] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"2a4ff939eb86b085b2f7503c","created_at":"2026-10-02T14:13:07.938Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51887796854ccc924035e58756e2b861","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:13:07.952Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-2a4ff939eb86b085b2f7503c: done: 0.043428 sec","job_status":"done","queue_duration_s":1.149131,"scheduling_latency_s":1.149159,"redis_calls":10,"redis_duration_s":0.0031460000000000004,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000401,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000666,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00080042,"duration_s":0.043428,"completed_at":"2026-10-02T14:13:09.145Z","load_balancing_strategy":"primary","db_duration_s":0.004168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1274gitlab # [ 231.410661] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bdc1eebf3bba7b573b10c3d8","created_at":"2026-10-02T14:13:07.961Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eeaafdfa8d04e29e346da130167c928e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.970Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bdc1eebf3bba7b573b10c3d8: start","job_status":"start","queue_duration_s":1.176868,"scheduling_latency_s":1.176901}1275gitlab # [ 231.427333] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.183Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bdc1eebf3bba7b573b10c3d8","created_at":"2026-10-02T14:13:07.961Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eeaafdfa8d04e29e346da130167c928e","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:07.970Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bdc1eebf3bba7b573b10c3d8: done: 0.036653 sec","job_status":"done","queue_duration_s":1.176868,"scheduling_latency_s":1.176901,"redis_calls":12,"redis_duration_s":0.003369,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001621,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00072938,"exclusive_lock_hold_duration_s":0.0052869399999999995,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.036653,"completed_at":"2026-10-02T14:13:09.183Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002475,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 231.476809] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cfb812e62ece2b93b364f5b4","created_at":"2026-10-02T14:13:07.991Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1bd109c45c5d64af8d44dc94bda0f88e","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.009Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cfb812e62ece2b93b364f5b4: start","job_status":"start","queue_duration_s":1.176593,"scheduling_latency_s":1.17663}1277gitlab # [ 231.713802] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.620Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"cfb812e62ece2b93b364f5b4","created_at":"2026-10-02T14:13:07.991Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1bd109c45c5d64af8d44dc94bda0f88e","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.009Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-cfb812e62ece2b93b364f5b4: done: 0.434911 sec","job_status":"done","queue_duration_s":1.176593,"scheduling_latency_s":1.17663,"redis_calls":565,"redis_duration_s":0.09194,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000495,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001247,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.090198,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.388042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031688,"duration_s":0.434911,"completed_at":"2026-10-02T14:13:09.620Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000706,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1278gitlab # [ 231.781356] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"346d58ee347575b4a30ce552","created_at":"2026-10-02T14:13:08.039Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a12a653a756c5eef80f6186d8196e3ec","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.059Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-346d58ee347575b4a30ce552: start","job_status":"start","queue_duration_s":1.563049,"scheduling_latency_s":1.563082}1279gitlab # [ 231.802377] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.666Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"346d58ee347575b4a30ce552","created_at":"2026-10-02T14:13:08.039Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a12a653a756c5eef80f6186d8196e3ec","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.059Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-346d58ee347575b4a30ce552: done: 0.043793 sec","job_status":"done","queue_duration_s":1.563049,"scheduling_latency_s":1.563082,"redis_calls":13,"redis_duration_s":0.002848,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000459,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001404,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01366434,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.043793,"completed_at":"2026-10-02T14:13:09.665Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004326,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1280gitlab # [ 231.855727] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.668Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e96219dd33949526281c0052","created_at":"2026-10-02T14:13:08.065Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34dd85b8a74f8f9261f3c88dd1dcdedb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.082Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e96219dd33949526281c0052: start","job_status":"start","queue_duration_s":1.585772,"scheduling_latency_s":1.585809}1281gitlab # [ 231.871342] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e96219dd33949526281c0052","created_at":"2026-10-02T14:13:08.065Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34dd85b8a74f8f9261f3c88dd1dcdedb","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.082Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e96219dd33949526281c0052: done: 0.026015 sec","job_status":"done","queue_duration_s":1.585772,"scheduling_latency_s":1.585809,"redis_calls":10,"redis_duration_s":0.002086,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000407,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054388,"duration_s":0.026015,"completed_at":"2026-10-02T14:13:09.693Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000564,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 231.916273] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd36c324d5166f291ff09d23","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd36c324d5166f291ff09d23: start","job_status":"start","queue_duration_s":1.283732,"scheduling_latency_s":1.283769}1283gitlab # [ 231.929292] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd36c324d5166f291ff09d23","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd36c324d5166f291ff09d23: done: 0.030611 sec","job_status":"done","queue_duration_s":1.283732,"scheduling_latency_s":1.283769,"redis_calls":16,"redis_duration_s":0.0032,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000431,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00182,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003506,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030611,"completed_at":"2026-10-02T14:13:09.726Z","load_balancing_strategy":"primary","db_duration_s":0.000577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1284gitlab # [ 231.972409] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c3bc2a22ed6dd56a12a1eb0","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c3bc2a22ed6dd56a12a1eb0: start","job_status":"start","queue_duration_s":1.31604,"scheduling_latency_s":1.316077}1285gitlab # [ 231.987049] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6c3bc2a22ed6dd56a12a1eb0","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6c3bc2a22ed6dd56a12a1eb0: done: 0.021037 sec","job_status":"done","queue_duration_s":1.31604,"scheduling_latency_s":1.316077,"redis_calls":13,"redis_duration_s":0.0022519999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001388,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021037,"completed_at":"2026-10-02T14:13:09.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 232.029804] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c52a5a14fefc4e7c6b86794","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c52a5a14fefc4e7c6b86794: start","job_status":"start","queue_duration_s":1.338769,"scheduling_latency_s":1.338804}1287gitlab # [ 232.042831] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c52a5a14fefc4e7c6b86794","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c52a5a14fefc4e7c6b86794: done: 0.021398 sec","job_status":"done","queue_duration_s":1.338769,"scheduling_latency_s":1.338804,"redis_calls":13,"redis_duration_s":0.00236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001484,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021398,"completed_at":"2026-10-02T14:13:09.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 232.086176] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.774Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8197a4f521047876e240adf0","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8197a4f521047876e240adf0: start","job_status":"start","queue_duration_s":1.362068,"scheduling_latency_s":1.362104}1289gitlab # [ 232.099947] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8197a4f521047876e240adf0","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8197a4f521047876e240adf0: done: 0.020909 sec","job_status":"done","queue_duration_s":1.362068,"scheduling_latency_s":1.362104,"redis_calls":13,"redis_duration_s":0.0023250000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000854,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001471,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020909,"completed_at":"2026-10-02T14:13:09.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 232.143660] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17726038d8f597a36404efed","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17726038d8f597a36404efed: start","job_status":"start","queue_duration_s":1.385119,"scheduling_latency_s":1.385158}1291gitlab # [ 232.157914] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17726038d8f597a36404efed","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17726038d8f597a36404efed: done: 0.021292 sec","job_status":"done","queue_duration_s":1.385119,"scheduling_latency_s":1.385158,"redis_calls":13,"redis_duration_s":0.0022979999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021292,"completed_at":"2026-10-02T14:13:09.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 232.201850] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd7cc8eaa4f40acf1c2015d","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd7cc8eaa4f40acf1c2015d: start","job_status":"start","queue_duration_s":1.40835,"scheduling_latency_s":1.40839}1293gitlab # [ 232.215555] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd7cc8eaa4f40acf1c2015d","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd7cc8eaa4f40acf1c2015d: done: 0.021035 sec","job_status":"done","queue_duration_s":1.40835,"scheduling_latency_s":1.40839,"redis_calls":13,"redis_duration_s":0.00232,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021035,"completed_at":"2026-10-02T14:13:09.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 232.257441] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b147df506cc4eb7fe0832628","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b147df506cc4eb7fe0832628: start","job_status":"start","queue_duration_s":1.431304,"scheduling_latency_s":1.431341}1295gitlab # [ 232.270617] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b147df506cc4eb7fe0832628","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b147df506cc4eb7fe0832628: done: 0.021427 sec","job_status":"done","queue_duration_s":1.431304,"scheduling_latency_s":1.431341,"redis_calls":13,"redis_duration_s":0.0024720000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021427,"completed_at":"2026-10-02T14:13:09.864Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 232.312320] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4ec082b6b1d166ad953033","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4ec082b6b1d166ad953033: start","job_status":"start","queue_duration_s":1.454714,"scheduling_latency_s":1.454753}1297gitlab # [ 232.325345] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d4ec082b6b1d166ad953033","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d4ec082b6b1d166ad953033: done: 0.021709 sec","job_status":"done","queue_duration_s":1.454714,"scheduling_latency_s":1.454753,"redis_calls":13,"redis_duration_s":0.002435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000942,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021709,"completed_at":"2026-10-02T14:13:09.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 232.366426] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0aca2847e2aadd7cc4de889","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0aca2847e2aadd7cc4de889: start","job_status":"start","queue_duration_s":1.478086,"scheduling_latency_s":1.478122}1299gitlab # [ 232.379570] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0aca2847e2aadd7cc4de889","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0aca2847e2aadd7cc4de889: done: 0.02187 sec","job_status":"done","queue_duration_s":1.478086,"scheduling_latency_s":1.478122,"redis_calls":13,"redis_duration_s":0.002413,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000856,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001557,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02187,"completed_at":"2026-10-02T14:13:09.912Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 232.422774] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"492da460be1b7e420fad7316","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-492da460be1b7e420fad7316: start","job_status":"start","queue_duration_s":1.501611,"scheduling_latency_s":1.501648}1301gitlab # [ 232.436186] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.936Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"492da460be1b7e420fad7316","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-492da460be1b7e420fad7316: done: 0.023083 sec","job_status":"done","queue_duration_s":1.501611,"scheduling_latency_s":1.501648,"redis_calls":13,"redis_duration_s":0.00256,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001561,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014668,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023083,"completed_at":"2026-10-02T14:13:09.936Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 232.477540] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29569faf542ddfb787af3bb5","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29569faf542ddfb787af3bb5: start","job_status":"start","queue_duration_s":1.526782,"scheduling_latency_s":1.526821}1303gitlab # [ 232.490642] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29569faf542ddfb787af3bb5","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29569faf542ddfb787af3bb5: done: 0.025007 sec","job_status":"done","queue_duration_s":1.526782,"scheduling_latency_s":1.526821,"redis_calls":13,"redis_duration_s":0.004132,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003103,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025007,"completed_at":"2026-10-02T14:13:09.963Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 232.532197] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.965Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74b72c4836e7ec18d82c9f48","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74b72c4836e7ec18d82c9f48: start","job_status":"start","queue_duration_s":1.55371,"scheduling_latency_s":1.553749}1305gitlab # [ 232.545124] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:09.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74b72c4836e7ec18d82c9f48","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74b72c4836e7ec18d82c9f48: done: 0.033995 sec","job_status":"done","queue_duration_s":1.55371,"scheduling_latency_s":1.553749,"redis_calls":13,"redis_duration_s":0.007631,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002085,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005546,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033995,"completed_at":"2026-10-02T14:13:09.999Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 232.586222] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.002Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2566bc17589e7f056ecca542","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2566bc17589e7f056ecca542: start","job_status":"start","queue_duration_s":1.589954,"scheduling_latency_s":1.59}1307gitlab # [ 232.600423] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2566bc17589e7f056ecca542","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2566bc17589e7f056ecca542: done: 0.028128 sec","job_status":"done","queue_duration_s":1.589954,"scheduling_latency_s":1.59,"redis_calls":13,"redis_duration_s":0.00283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001764,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028128,"completed_at":"2026-10-02T14:13:10.030Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 232.641483] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb674e6bd9ea582769cbfd08","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb674e6bd9ea582769cbfd08: start","job_status":"start","queue_duration_s":1.620139,"scheduling_latency_s":1.620226}1309gitlab # [ 232.654443] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb674e6bd9ea582769cbfd08","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb674e6bd9ea582769cbfd08: done: 0.027673 sec","job_status":"done","queue_duration_s":1.620139,"scheduling_latency_s":1.620226,"redis_calls":13,"redis_duration_s":0.003058,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001267,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001791,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027673,"completed_at":"2026-10-02T14:13:10.059Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 232.695613] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01265063cc15096e0d5542ce","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01265063cc15096e0d5542ce: start","job_status":"start","queue_duration_s":1.650872,"scheduling_latency_s":1.650923}1311gitlab # [ 232.708759] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01265063cc15096e0d5542ce","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01265063cc15096e0d5542ce: done: 0.025905 sec","job_status":"done","queue_duration_s":1.650872,"scheduling_latency_s":1.650923,"redis_calls":13,"redis_duration_s":0.002622,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001732,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025905,"completed_at":"2026-10-02T14:13:10.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 232.749721] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e4149e24adf953af4b0e518","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e4149e24adf953af4b0e518: start","job_status":"start","queue_duration_s":1.678655,"scheduling_latency_s":1.678695}1313gitlab # [ 232.762734] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e4149e24adf953af4b0e518","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e4149e24adf953af4b0e518: done: 0.021201 sec","job_status":"done","queue_duration_s":1.678655,"scheduling_latency_s":1.678695,"redis_calls":13,"redis_duration_s":0.002306,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001488,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021201,"completed_at":"2026-10-02T14:13:10.111Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 232.804978] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13b9ff64138d01f9177b5f2d","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13b9ff64138d01f9177b5f2d: start","job_status":"start","queue_duration_s":1.701557,"scheduling_latency_s":1.701593}1315gitlab # [ 232.818061] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13b9ff64138d01f9177b5f2d","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13b9ff64138d01f9177b5f2d: done: 0.018287 sec","job_status":"done","queue_duration_s":1.701557,"scheduling_latency_s":1.701593,"redis_calls":13,"redis_duration_s":0.0021190000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001426,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018287,"completed_at":"2026-10-02T14:13:10.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 232.859782] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f7b71d0197f73cb3cf41f75","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f7b71d0197f73cb3cf41f75: start","job_status":"start","queue_duration_s":1.721499,"scheduling_latency_s":1.721532}1317gitlab # [ 232.873072] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f7b71d0197f73cb3cf41f75","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f7b71d0197f73cb3cf41f75: done: 0.017454 sec","job_status":"done","queue_duration_s":1.721499,"scheduling_latency_s":1.721532,"redis_calls":13,"redis_duration_s":0.002283,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001594,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017454,"completed_at":"2026-10-02T14:13:10.151Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 232.914220] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82b32325ad3ef55517110c6b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82b32325ad3ef55517110c6b: start","job_status":"start","queue_duration_s":1.740636,"scheduling_latency_s":1.740671}1319gitlab # [ 232.927389] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82b32325ad3ef55517110c6b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82b32325ad3ef55517110c6b: done: 0.017421 sec","job_status":"done","queue_duration_s":1.740636,"scheduling_latency_s":1.740671,"redis_calls":13,"redis_duration_s":0.0018909999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017421,"completed_at":"2026-10-02T14:13:10.169Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 232.969176] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05052cd523d6c47081b375ce","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05052cd523d6c47081b375ce: start","job_status":"start","queue_duration_s":1.759267,"scheduling_latency_s":1.759296}1321gitlab # [ 232.982345] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05052cd523d6c47081b375ce","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05052cd523d6c47081b375ce: done: 0.017431 sec","job_status":"done","queue_duration_s":1.759267,"scheduling_latency_s":1.759296,"redis_calls":13,"redis_duration_s":0.001938,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000688,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017431,"completed_at":"2026-10-02T14:13:10.188Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 233.025924] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d04055038671a18a8a9b2e3b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d04055038671a18a8a9b2e3b: start","job_status":"start","queue_duration_s":1.778196,"scheduling_latency_s":1.778227}1323gitlab # [ 233.046479] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d04055038671a18a8a9b2e3b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d04055038671a18a8a9b2e3b: done: 0.017719 sec","job_status":"done","queue_duration_s":1.778196,"scheduling_latency_s":1.778227,"redis_calls":13,"redis_duration_s":0.002215,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017719,"completed_at":"2026-10-02T14:13:10.207Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 233.108349] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"607dc70a30e80f6288ae0d14","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-607dc70a30e80f6288ae0d14: start","job_status":"start","queue_duration_s":1.797368,"scheduling_latency_s":1.7974}1325gitlab # [ 233.126313] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"607dc70a30e80f6288ae0d14","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-607dc70a30e80f6288ae0d14: done: 0.017498 sec","job_status":"done","queue_duration_s":1.797368,"scheduling_latency_s":1.7974,"redis_calls":13,"redis_duration_s":0.001948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017498,"completed_at":"2026-10-02T14:13:10.226Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 233.173655] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.228Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"035247bb64340a0c83d64777","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-035247bb64340a0c83d64777: start","job_status":"start","queue_duration_s":1.816421,"scheduling_latency_s":1.816454}1327gitlab # [ 233.186677] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"035247bb64340a0c83d64777","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-035247bb64340a0c83d64777: done: 0.017484 sec","job_status":"done","queue_duration_s":1.816421,"scheduling_latency_s":1.816454,"redis_calls":13,"redis_duration_s":0.002069,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017484,"completed_at":"2026-10-02T14:13:10.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 233.228922] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56b79d9cb5f266c7a7d8a558","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56b79d9cb5f266c7a7d8a558: start","job_status":"start","queue_duration_s":1.835454,"scheduling_latency_s":1.835488}1329gitlab # [ 233.241898] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56b79d9cb5f266c7a7d8a558","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56b79d9cb5f266c7a7d8a558: done: 0.018329 sec","job_status":"done","queue_duration_s":1.835454,"scheduling_latency_s":1.835488,"redis_calls":13,"redis_duration_s":0.002189,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001425,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018329,"completed_at":"2026-10-02T14:13:10.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 233.282734] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"374585aedf369746771a752c","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-374585aedf369746771a752c: start","job_status":"start","queue_duration_s":1.855522,"scheduling_latency_s":1.855555}1331gitlab # [ 233.295824] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"374585aedf369746771a752c","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-374585aedf369746771a752c: done: 0.018209 sec","job_status":"done","queue_duration_s":1.855522,"scheduling_latency_s":1.855555,"redis_calls":13,"redis_duration_s":0.002091,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000736,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001355,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018209,"completed_at":"2026-10-02T14:13:10.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 233.337166] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8940fa8e6b320dabdd2e98a","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8940fa8e6b320dabdd2e98a: start","job_status":"start","queue_duration_s":1.875534,"scheduling_latency_s":1.875568}1333gitlab # [ 233.350033] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8940fa8e6b320dabdd2e98a","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8940fa8e6b320dabdd2e98a: done: 0.018257 sec","job_status":"done","queue_duration_s":1.875534,"scheduling_latency_s":1.875568,"redis_calls":13,"redis_duration_s":0.002124,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018257,"completed_at":"2026-10-02T14:13:10.305Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 233.390994] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbf5be848a40dc6af1701e09","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbf5be848a40dc6af1701e09: start","job_status":"start","queue_duration_s":1.895567,"scheduling_latency_s":1.895601}1335gitlab # [ 233.404111] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbf5be848a40dc6af1701e09","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbf5be848a40dc6af1701e09: done: 0.019889 sec","job_status":"done","queue_duration_s":1.895567,"scheduling_latency_s":1.895601,"redis_calls":13,"redis_duration_s":0.003103,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019889,"completed_at":"2026-10-02T14:13:10.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 233.446379] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee40af9acef4648ce02d073b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee40af9acef4648ce02d073b: start","job_status":"start","queue_duration_s":1.917315,"scheduling_latency_s":1.917352}1337gitlab # [ 233.459847] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee40af9acef4648ce02d073b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee40af9acef4648ce02d073b: done: 0.019133 sec","job_status":"done","queue_duration_s":1.917315,"scheduling_latency_s":1.917352,"redis_calls":13,"redis_duration_s":0.00238,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019133,"completed_at":"2026-10-02T14:13:10.348Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 233.501612] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3106f8194a9780e43d2a77e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3106f8194a9780e43d2a77e: start","job_status":"start","queue_duration_s":1.938217,"scheduling_latency_s":1.938252}1339gitlab # [ 233.514824] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3106f8194a9780e43d2a77e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3106f8194a9780e43d2a77e: done: 0.018535 sec","job_status":"done","queue_duration_s":1.938217,"scheduling_latency_s":1.938252,"redis_calls":13,"redis_duration_s":0.002193,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000773,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018535,"completed_at":"2026-10-02T14:13:10.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 233.555712] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"401c34ebc9119f2e0986c5bf","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-401c34ebc9119f2e0986c5bf: start","job_status":"start","queue_duration_s":1.95827,"scheduling_latency_s":1.958303}1341gitlab # [ 233.568952] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"401c34ebc9119f2e0986c5bf","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-401c34ebc9119f2e0986c5bf: done: 0.018932 sec","job_status":"done","queue_duration_s":1.95827,"scheduling_latency_s":1.958303,"redis_calls":13,"redis_duration_s":0.002174,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001422,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018932,"completed_at":"2026-10-02T14:13:10.389Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 233.609698] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a22d0fcc7670591987aba17e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a22d0fcc7670591987aba17e: start","job_status":"start","queue_duration_s":1.979005,"scheduling_latency_s":1.979038}1343gitlab # [ 233.623737] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a22d0fcc7670591987aba17e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a22d0fcc7670591987aba17e: done: 0.019322 sec","job_status":"done","queue_duration_s":1.979005,"scheduling_latency_s":1.979038,"redis_calls":13,"redis_duration_s":0.002219,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001441,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019322,"completed_at":"2026-10-02T14:13:10.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 233.664985] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.412Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29a55cadce156eb0ffaaf878","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29a55cadce156eb0ffaaf878: start","job_status":"start","queue_duration_s":1.999919,"scheduling_latency_s":1.999953}1345gitlab # [ 233.685928] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29a55cadce156eb0ffaaf878","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29a55cadce156eb0ffaaf878: done: 0.020691 sec","job_status":"done","queue_duration_s":1.999919,"scheduling_latency_s":1.999953,"redis_calls":13,"redis_duration_s":0.003776,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000845,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020691,"completed_at":"2026-10-02T14:13:10.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 233.746324] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"594abf40c19c9701c67f599a","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-594abf40c19c9701c67f599a: start","job_status":"start","queue_duration_s":2.022186,"scheduling_latency_s":2.02222}1347gitlab # [ 233.760816] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"594abf40c19c9701c67f599a","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-594abf40c19c9701c67f599a: done: 0.020373 sec","job_status":"done","queue_duration_s":2.022186,"scheduling_latency_s":2.02222,"redis_calls":13,"redis_duration_s":0.0026839999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020373,"completed_at":"2026-10-02T14:13:10.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 233.802312] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef6eeb1d0bf99e922429a233","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef6eeb1d0bf99e922429a233: start","job_status":"start","queue_duration_s":2.044293,"scheduling_latency_s":2.044324}1349gitlab # [ 233.815617] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef6eeb1d0bf99e922429a233","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef6eeb1d0bf99e922429a233: done: 0.016535 sec","job_status":"done","queue_duration_s":2.044293,"scheduling_latency_s":2.044324,"redis_calls":13,"redis_duration_s":0.0019190000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001275,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016535,"completed_at":"2026-10-02T14:13:10.472Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 233.859760] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.474Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc258cd01984385c079e722","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc258cd01984385c079e722: start","job_status":"start","queue_duration_s":2.06194,"scheduling_latency_s":2.061958}1351gitlab # [ 233.879939] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9dc258cd01984385c079e722","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9dc258cd01984385c079e722: done: 0.015314 sec","job_status":"done","queue_duration_s":2.06194,"scheduling_latency_s":2.061958,"redis_calls":13,"redis_duration_s":0.001824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015314,"completed_at":"2026-10-02T14:13:10.489Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 233.939553] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edb5a8f59ae5156492252f43","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edb5a8f59ae5156492252f43: start","job_status":"start","queue_duration_s":2.078592,"scheduling_latency_s":2.078609}1353gitlab # [ 233.960895] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edb5a8f59ae5156492252f43","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edb5a8f59ae5156492252f43: done: 0.015495 sec","job_status":"done","queue_duration_s":2.078592,"scheduling_latency_s":2.078609,"redis_calls":13,"redis_duration_s":0.001992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015495,"completed_at":"2026-10-02T14:13:10.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 234.017941] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8e04feeeae705a8e7771b0e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8e04feeeae705a8e7771b0e: start","job_status":"start","queue_duration_s":2.095419,"scheduling_latency_s":2.095437}1355gitlab # [ 234.031901] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8e04feeeae705a8e7771b0e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8e04feeeae705a8e7771b0e: done: 0.015747 sec","job_status":"done","queue_duration_s":2.095419,"scheduling_latency_s":2.095437,"redis_calls":13,"redis_duration_s":0.001882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001231,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015747,"completed_at":"2026-10-02T14:13:10.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 234.075729] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54183792dba18c03d6f1dd86","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54183792dba18c03d6f1dd86: start","job_status":"start","queue_duration_s":2.112496,"scheduling_latency_s":2.112515}1357gitlab # [ 234.089534] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54183792dba18c03d6f1dd86","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54183792dba18c03d6f1dd86: done: 0.015303 sec","job_status":"done","queue_duration_s":2.112496,"scheduling_latency_s":2.112515,"redis_calls":13,"redis_duration_s":0.0018739999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015303,"completed_at":"2026-10-02T14:13:10.539Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 234.131170] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"635268b1f4cc4222e2e64472","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-635268b1f4cc4222e2e64472: start","job_status":"start","queue_duration_s":2.129229,"scheduling_latency_s":2.129247}1359gitlab # [ 234.144299] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"635268b1f4cc4222e2e64472","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-635268b1f4cc4222e2e64472: done: 0.015994 sec","job_status":"done","queue_duration_s":2.129229,"scheduling_latency_s":2.129247,"redis_calls":13,"redis_duration_s":0.002223,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001528,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015994,"completed_at":"2026-10-02T14:13:10.557Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 234.185240] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e32616c067f8f244a33d4f6","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e32616c067f8f244a33d4f6: start","job_status":"start","queue_duration_s":2.146615,"scheduling_latency_s":2.146635}1361gitlab # [ 234.198163] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e32616c067f8f244a33d4f6","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e32616c067f8f244a33d4f6: done: 0.015659 sec","job_status":"done","queue_duration_s":2.146615,"scheduling_latency_s":2.146635,"redis_calls":13,"redis_duration_s":0.001896,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000686,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00121,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015659,"completed_at":"2026-10-02T14:13:10.574Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 234.239363] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.575Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87d837dbf2741d7d81bf4f05","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87d837dbf2741d7d81bf4f05: start","job_status":"start","queue_duration_s":2.163421,"scheduling_latency_s":2.16344}1363gitlab # [ 234.253477] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87d837dbf2741d7d81bf4f05","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87d837dbf2741d7d81bf4f05: done: 0.015696 sec","job_status":"done","queue_duration_s":2.163421,"scheduling_latency_s":2.16344,"redis_calls":13,"redis_duration_s":0.002116,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001453,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015696,"completed_at":"2026-10-02T14:13:10.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 234.294660] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.592Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90471676df566e694c1d702e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90471676df566e694c1d702e: start","job_status":"start","queue_duration_s":2.180338,"scheduling_latency_s":2.180358}1365gitlab # [ 234.307754] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90471676df566e694c1d702e","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90471676df566e694c1d702e: done: 0.015692 sec","job_status":"done","queue_duration_s":2.180338,"scheduling_latency_s":2.180358,"redis_calls":13,"redis_duration_s":0.002059,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001407,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010684,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015692,"completed_at":"2026-10-02T14:13:10.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 234.349070] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7e5d2808787bcc25ff234d0","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7e5d2808787bcc25ff234d0: start","job_status":"start","queue_duration_s":2.197213,"scheduling_latency_s":2.19723}1367gitlab # [ 234.362020] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a7e5d2808787bcc25ff234d0","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a7e5d2808787bcc25ff234d0: done: 0.015848 sec","job_status":"done","queue_duration_s":2.197213,"scheduling_latency_s":2.19723,"redis_calls":13,"redis_duration_s":0.001957,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000662,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015848,"completed_at":"2026-10-02T14:13:10.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 234.403403] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c44c34986b1991d22a66390b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c44c34986b1991d22a66390b: start","job_status":"start","queue_duration_s":2.214401,"scheduling_latency_s":2.214419}1369gitlab # [ 234.416479] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c44c34986b1991d22a66390b","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c44c34986b1991d22a66390b: done: 0.015732 sec","job_status":"done","queue_duration_s":2.214401,"scheduling_latency_s":2.214419,"redis_calls":13,"redis_duration_s":0.001919,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001259,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015732,"completed_at":"2026-10-02T14:13:10.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 234.459368] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1328b52b9714c345fa6b32d6","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1328b52b9714c345fa6b32d6: start","job_status":"start","queue_duration_s":2.231305,"scheduling_latency_s":2.231322}1371gitlab # [ 234.472918] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1328b52b9714c345fa6b32d6","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1328b52b9714c345fa6b32d6: done: 0.015621 sec","job_status":"done","queue_duration_s":2.231305,"scheduling_latency_s":2.231322,"redis_calls":13,"redis_duration_s":0.0019950000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001294,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010579,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015621,"completed_at":"2026-10-02T14:13:10.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 234.514475] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf3c98bd57ddd76ff2f7fdb5","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf3c98bd57ddd76ff2f7fdb5: start","job_status":"start","queue_duration_s":2.248505,"scheduling_latency_s":2.248525}1373gitlab # [ 234.527901] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf3c98bd57ddd76ff2f7fdb5","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf3c98bd57ddd76ff2f7fdb5: done: 0.01592 sec","job_status":"done","queue_duration_s":2.248505,"scheduling_latency_s":2.248525,"redis_calls":13,"redis_duration_s":0.0020039999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001297,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01592,"completed_at":"2026-10-02T14:13:10.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 234.571054] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4037eaa3e65f231072b0ebbc","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4037eaa3e65f231072b0ebbc: start","job_status":"start","queue_duration_s":2.265984,"scheduling_latency_s":2.266007}1375gitlab # [ 234.584780] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4037eaa3e65f231072b0ebbc","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4037eaa3e65f231072b0ebbc: done: 0.016336 sec","job_status":"done","queue_duration_s":2.265984,"scheduling_latency_s":2.266007,"redis_calls":13,"redis_duration_s":0.002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001291,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011226,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016336,"completed_at":"2026-10-02T14:13:10.694Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 234.625917] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"855c070e9b005b559a85dd6d","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-855c070e9b005b559a85dd6d: start","job_status":"start","queue_duration_s":2.28362,"scheduling_latency_s":2.283638}1377gitlab # [ 234.639349] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"855c070e9b005b559a85dd6d","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-855c070e9b005b559a85dd6d: done: 0.016929 sec","job_status":"done","queue_duration_s":2.28362,"scheduling_latency_s":2.283638,"redis_calls":13,"redis_duration_s":0.002284,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001484,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016929,"completed_at":"2026-10-02T14:13:10.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 234.682867] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e09785b1d162dd16223670b7","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e09785b1d162dd16223670b7: start","job_status":"start","queue_duration_s":2.30201,"scheduling_latency_s":2.30203}1379gitlab # [ 234.696977] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e09785b1d162dd16223670b7","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e09785b1d162dd16223670b7: done: 0.016937 sec","job_status":"done","queue_duration_s":2.30201,"scheduling_latency_s":2.30203,"redis_calls":13,"redis_duration_s":0.002239,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001439,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016937,"completed_at":"2026-10-02T14:13:10.731Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 234.740250] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb6d53d25953cfb8aa15d2b1","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb6d53d25953cfb8aa15d2b1: start","job_status":"start","queue_duration_s":2.320495,"scheduling_latency_s":2.320515}1381gitlab # [ 234.753064] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:13:10.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb6d53d25953cfb8aa15d2b1","created_at":"2026-10-02T14:13:08.276Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-b8c83abd68e748c0","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2fccb30421680fea2f65e9e6a2366546","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:13:08.412Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb6d53d25953cfb8aa15d2b1: done: 0.017156 sec","job_status":"done","queue_duration_s":2.320495,"scheduling_latency_s":2.320515,"redis_calls":13,"redis_duration_s":0.002048,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017156,"completed_at":"2026-10-02T14:13:10.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 264.029314] setup-gitlab-start[2943]: W, [2026-10-02T14:13:41.935924 #2943] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1383gitlab # [ 283.359124] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"be1b634c8334a7662bb95bf2","created_at":"2026-10-02T14:14:01.249Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c3541fd9e88fe7931cde0a0099f9088","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.262Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-be1b634c8334a7662bb95bf2: start","job_status":"start","queue_duration_s":0.00265,"scheduling_latency_s":0.002701}1384gitlab # [ 283.517798] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.423Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"be1b634c8334a7662bb95bf2","created_at":"2026-10-02T14:14:01.249Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c3541fd9e88fe7931cde0a0099f9088","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.262Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-be1b634c8334a7662bb95bf2: done: 0.158526 sec","job_status":"done","queue_duration_s":0.00265,"scheduling_latency_s":0.002701,"redis_calls":17,"redis_duration_s":0.028161,"redis_read_bytes":907,"redis_write_bytes":1637,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.012506,"redis_feature_flag_read_bytes":900,"redis_feature_flag_write_bytes":468,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003368,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00244906,"exclusive_lock_hold_duration_s":0.03259632,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.158526,"completed_at":"2026-10-02T14:14:01.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1385gitlab # [ 283.581252] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.429Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a89e10ce1880c4ee7c0ed2c3","created_at":"2026-10-02T14:14:01.288Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3772966c76f506d8d22b93b39e14148","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.298Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a89e10ce1880c4ee7c0ed2c3: start","job_status":"start","queue_duration_s":0.130301,"scheduling_latency_s":0.130352}1386gitlab # [ 283.643124] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.550Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1387gitlab # [ 283.674177] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a89e10ce1880c4ee7c0ed2c3","created_at":"2026-10-02T14:14:01.288Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3772966c76f506d8d22b93b39e14148","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.298Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a89e10ce1880c4ee7c0ed2c3: done: 0.152079 sec","job_status":"done","queue_duration_s":0.130301,"scheduling_latency_s":0.130352,"redis_calls":7,"redis_duration_s":0.016306,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.011698,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004608,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.152079,"completed_at":"2026-10-02T14:14:01.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 283.740517] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.587Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4387ed2e90ef363d57d65cc1","created_at":"2026-10-02T14:14:01.316Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.325Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4387ed2e90ef363d57d65cc1: start","job_status":"start","queue_duration_s":0.261233,"scheduling_latency_s":0.261274}1389gitlab # [ 283.827069] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.733Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"4387ed2e90ef363d57d65cc1","created_at":"2026-10-02T14:14:01.316Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.325Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-4387ed2e90ef363d57d65cc1: done: 0.146223 sec","job_status":"done","queue_duration_s":0.261233,"scheduling_latency_s":0.261274,"redis_calls":15,"redis_duration_s":0.041171000000000006,"redis_read_bytes":144,"redis_write_bytes":4079,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_calls":2,"redis_queues_duration_s":0.002571,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.035014,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003114,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.146223,"completed_at":"2026-10-02T14:14:01.733Z","load_balancing_strategy":"primary","db_duration_s":0.008284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 283.897188] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.738Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e3fc88f8e24909e702da6063","created_at":"2026-10-02T14:14:01.334Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd4a9d8552881116754010b4bcd1dc08","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.390Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e3fc88f8e24909e702da6063: start","job_status":"start","queue_duration_s":0.348469,"scheduling_latency_s":0.348521}1391gitlab # [ 283.978160] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e3fc88f8e24909e702da6063","created_at":"2026-10-02T14:14:01.334Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd4a9d8552881116754010b4bcd1dc08","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.390Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e3fc88f8e24909e702da6063: done: 0.146027 sec","job_status":"done","queue_duration_s":0.348469,"scheduling_latency_s":0.348521,"redis_calls":12,"redis_duration_s":0.03394,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004367,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.013301,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00382884,"exclusive_lock_hold_duration_s":0.02054404,"duration_s":0.146027,"completed_at":"2026-10-02T14:14:01.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1392gitlab # [ 284.045908] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:01.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"056b4b1a8cf42dc9ad91ce2a","created_at":"2026-10-02T14:14:01.410Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cfa1a69c8bf8903ff0d6d4bdb51cfab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.418Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-056b4b1a8cf42dc9ad91ce2a: start","job_status":"start","queue_duration_s":0.504009,"scheduling_latency_s":0.504067}1393gitlab # [ 284.204026] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.110Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"056b4b1a8cf42dc9ad91ce2a","created_at":"2026-10-02T14:14:01.410Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cfa1a69c8bf8903ff0d6d4bdb51cfab","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.418Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-056b4b1a8cf42dc9ad91ce2a: done: 0.187857 sec","job_status":"done","queue_duration_s":0.504009,"scheduling_latency_s":0.504067,"redis_calls":9,"redis_duration_s":0.012386000000000001,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005589,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00454642,"exclusive_lock_hold_duration_s":0.1304093,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.187857,"completed_at":"2026-10-02T14:14:02.110Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.120049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 284.272149] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.113Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0c0d0830ae4c0caad5352c19","created_at":"2026-10-02T14:14:01.436Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a1190dfabcf832fb2fe65b31d27d5dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.444Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0c0d0830ae4c0caad5352c19: start","job_status":"start","queue_duration_s":0.669084,"scheduling_latency_s":0.669132}1395gitlab # [ 284.289828] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.130Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1396gitlab # [ 284.293768] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.141Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"0c0d0830ae4c0caad5352c19","created_at":"2026-10-02T14:14:01.436Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a1190dfabcf832fb2fe65b31d27d5dd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.444Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-0c0d0830ae4c0caad5352c19: done: 0.028804 sec","job_status":"done","queue_duration_s":0.669084,"scheduling_latency_s":0.669132,"redis_calls":7,"redis_duration_s":0.002283,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001673,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00061,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028804,"completed_at":"2026-10-02T14:14:02.141Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1397gitlab # [ 284.337180] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.144Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"935b69cf0988e3ee6a359e9b","created_at":"2026-10-02T14:14:01.489Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71819422b1938fa1239a340fc706e640","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.494Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-935b69cf0988e3ee6a359e9b: start","job_status":"start","queue_duration_s":0.64951,"scheduling_latency_s":0.649557}1398gitlab # [ 284.352534] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.159Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1399gitlab # [ 284.356396] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"935b69cf0988e3ee6a359e9b","created_at":"2026-10-02T14:14:01.489Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"71819422b1938fa1239a340fc706e640","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.494Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-935b69cf0988e3ee6a359e9b: done: 0.025151 sec","job_status":"done","queue_duration_s":0.64951,"scheduling_latency_s":0.649557,"redis_calls":7,"redis_duration_s":0.001946,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025151,"completed_at":"2026-10-02T14:14:02.169Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 284.400579] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d9dd957c7649ee58c5e1c586","created_at":"2026-10-02T14:14:01.508Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.513Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d9dd957c7649ee58c5e1c586: start","job_status":"start","queue_duration_s":0.6586,"scheduling_latency_s":0.658642}1401gitlab # [ 284.425749] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d9dd957c7649ee58c5e1c586","created_at":"2026-10-02T14:14:01.508Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.513Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d9dd957c7649ee58c5e1c586: done: 0.160754 sec","job_status":"done","queue_duration_s":0.6586,"scheduling_latency_s":0.658642,"redis_calls":60,"redis_duration_s":0.018102,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000829,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.016489,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.160754,"completed_at":"2026-10-02T14:14:02.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 284.472323] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.334Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c49aae8dd77c34a4914bbb18","created_at":"2026-10-02T14:14:01.519Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da4000f7a818d9b9a52be263fc7cae50","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.524Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c49aae8dd77c34a4914bbb18: start","job_status":"start","queue_duration_s":0.80979,"scheduling_latency_s":0.809831}1403gitlab # [ 284.487359] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.368Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"c49aae8dd77c34a4914bbb18","created_at":"2026-10-02T14:14:01.519Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da4000f7a818d9b9a52be263fc7cae50","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.524Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-c49aae8dd77c34a4914bbb18: done: 0.033808 sec","job_status":"done","queue_duration_s":0.80979,"scheduling_latency_s":0.809831,"redis_calls":9,"redis_duration_s":0.0027280000000000004,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00144,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001288,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00085064,"exclusive_lock_hold_duration_s":0.00679226,"duration_s":0.033808,"completed_at":"2026-10-02T14:14:02.368Z","load_balancing_strategy":"primary","db_duration_s":0.001714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 284.532328] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.371Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b7d4df3f03092b157d8637ff","created_at":"2026-10-02T14:14:01.551Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ae90acd98db394eca2ccff6f5737a74","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.563Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b7d4df3f03092b157d8637ff: start","job_status":"start","queue_duration_s":0.808007,"scheduling_latency_s":0.808057}1405gitlab # [ 284.547307] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.422Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b7d4df3f03092b157d8637ff","created_at":"2026-10-02T14:14:01.551Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3ae90acd98db394eca2ccff6f5737a74","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.563Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b7d4df3f03092b157d8637ff: done: 0.051592 sec","job_status":"done","queue_duration_s":0.808007,"scheduling_latency_s":0.808057,"redis_calls":7,"redis_duration_s":0.008636,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00778,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000856,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.051592,"completed_at":"2026-10-02T14:14:02.422Z","load_balancing_strategy":"primary","db_duration_s":0.008768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 284.591412] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.425Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"93b19964fd6199e4cd466420","created_at":"2026-10-02T14:14:01.577Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ecc5057b4dd98c1a9b4731f2d00ac4d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.583Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-93b19964fd6199e4cd466420: start","job_status":"start","queue_duration_s":0.842467,"scheduling_latency_s":0.84252}1407gitlab # [ 284.606861] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.458Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"93b19964fd6199e4cd466420","created_at":"2026-10-02T14:14:01.577Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9ecc5057b4dd98c1a9b4731f2d00ac4d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.583Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-93b19964fd6199e4cd466420: done: 0.033119 sec","job_status":"done","queue_duration_s":0.842467,"scheduling_latency_s":0.84252,"redis_calls":8,"redis_duration_s":0.00263,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033119,"completed_at":"2026-10-02T14:14:02.458Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 284.651308] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a92765307371ac61ac04e54e","created_at":"2026-10-02T14:14:01.595Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a92d8d3d5a428b18f0423dcfe30b5c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:14:01.600Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a92765307371ac61ac04e54e: start","job_status":"start","queue_duration_s":0.860919,"scheduling_latency_s":0.860967}1409gitlab # [ 284.665632] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.476Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1410gitlab # [ 284.669592] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.486Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a92765307371ac61ac04e54e","created_at":"2026-10-02T14:14:01.595Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39a92d8d3d5a428b18f0423dcfe30b5c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:14:01.600Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a92765307371ac61ac04e54e: done: 0.024887 sec","job_status":"done","queue_duration_s":0.860919,"scheduling_latency_s":0.860967,"redis_calls":7,"redis_duration_s":0.0019219999999999999,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000578,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024887,"completed_at":"2026-10-02T14:14:02.486Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 284.711152] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.488Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5424045f3128748439bcb736","created_at":"2026-10-02T14:14:01.639Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4852316c2f140ff3217543d2c0613520","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.653Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5424045f3128748439bcb736: start","job_status":"start","queue_duration_s":0.835294,"scheduling_latency_s":0.835338}1412gitlab # [ 284.726183] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.525Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5424045f3128748439bcb736","created_at":"2026-10-02T14:14:01.639Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4852316c2f140ff3217543d2c0613520","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.653Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5424045f3128748439bcb736: done: 0.036476 sec","job_status":"done","queue_duration_s":0.835294,"scheduling_latency_s":0.835338,"redis_calls":7,"redis_duration_s":0.001995,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036476,"completed_at":"2026-10-02T14:14:02.525Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 284.768477] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.527Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e13a25d936ad5fbf1de716ef","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e13a25d936ad5fbf1de716ef: start","job_status":"start","queue_duration_s":0.832535,"scheduling_latency_s":0.832579}1414gitlab # [ 284.782342] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.548Z","message":"No existing merge request to be cleaned up."}1415gitlab # [ 284.784124] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.565Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e13a25d936ad5fbf1de716ef","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e13a25d936ad5fbf1de716ef: done: 0.038012 sec","job_status":"done","queue_duration_s":0.832535,"scheduling_latency_s":0.832579,"redis_calls":10,"redis_duration_s":0.002752,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001173,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026784,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038012,"completed_at":"2026-10-02T14:14:02.565Z","load_balancing_strategy":"primary","db_duration_s":0.003696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 284.824840] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.567Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2fa651020ad8b24c21ee43e8","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2fa651020ad8b24c21ee43e8: start","job_status":"start","queue_duration_s":0.872598,"scheduling_latency_s":0.872641}1417gitlab # [ 284.837858] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.582Z","message":"No existing merge request to be cleaned up."}1418gitlab # [ 284.839619] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.592Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2fa651020ad8b24c21ee43e8","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2fa651020ad8b24c21ee43e8: done: 0.025129 sec","job_status":"done","queue_duration_s":0.872598,"scheduling_latency_s":0.872641,"redis_calls":10,"redis_duration_s":0.001633,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025129,"completed_at":"2026-10-02T14:14:02.592Z","load_balancing_strategy":"primary","db_duration_s":0.002231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 284.880754] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.593Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0942401c00f81c8c1d79d6b4","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0942401c00f81c8c1d79d6b4: start","job_status":"start","queue_duration_s":0.899056,"scheduling_latency_s":0.899079}1420gitlab # [ 284.893499] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.606Z","message":"No existing merge request to be cleaned up."}1421gitlab # [ 284.895205] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.615Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0942401c00f81c8c1d79d6b4","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0942401c00f81c8c1d79d6b4: done: 0.021874 sec","job_status":"done","queue_duration_s":0.899056,"scheduling_latency_s":0.899079,"redis_calls":10,"redis_duration_s":0.001477,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000625,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021874,"completed_at":"2026-10-02T14:14:02.615Z","load_balancing_strategy":"primary","db_duration_s":0.001987,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 284.935535] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.617Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f5f2e1dbc3cee6522d8a0702","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f5f2e1dbc3cee6522d8a0702: start","job_status":"start","queue_duration_s":0.92236,"scheduling_latency_s":0.922382}1423gitlab # [ 284.948537] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.628Z","message":"No existing merge request to be cleaned up."}1424gitlab # [ 284.950285] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.637Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f5f2e1dbc3cee6522d8a0702","created_at":"2026-10-02T14:14:01.657Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-bdd9685c550945af","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"35e856bd764c4f48b1ee4ba16799b96a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.694Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f5f2e1dbc3cee6522d8a0702: done: 0.020812 sec","job_status":"done","queue_duration_s":0.92236,"scheduling_latency_s":0.922382,"redis_calls":10,"redis_duration_s":0.001471,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000867,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020812,"completed_at":"2026-10-02T14:14:02.637Z","load_balancing_strategy":"primary","db_duration_s":0.00174,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1425gitlab # [ 284.991442] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.639Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"1915a99b026fb49fe55db161","created_at":"2026-10-02T14:14:01.675Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5e706f34eb98d829e3d3830d7053339","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.702Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-1915a99b026fb49fe55db161: start","job_status":"start","queue_duration_s":0.93728,"scheduling_latency_s":0.937303}1426gitlab # [ 285.006621] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"1915a99b026fb49fe55db161","created_at":"2026-10-02T14:14:01.675Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f5e706f34eb98d829e3d3830d7053339","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.702Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-1915a99b026fb49fe55db161: done: 0.220098 sec","job_status":"done","queue_duration_s":0.93728,"scheduling_latency_s":0.937303,"redis_calls":12,"redis_duration_s":0.002193,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051322,"exclusive_lock_hold_duration_s":0.19674518000000002,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.220098,"completed_at":"2026-10-02T14:14:02.859Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015649,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1427gitlab # [ 285.052679] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.861Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0c6bd4fbfbd725dbd3de6433","created_at":"2026-10-02T14:14:01.723Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffe2cbe6ccd542f10473e37a6b2f6149","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.729Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0c6bd4fbfbd725dbd3de6433: start","job_status":"start","queue_duration_s":1.131528,"scheduling_latency_s":1.131559}1428gitlab # [ 285.067589] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.872Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1429gitlab # [ 285.071509] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.879Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0c6bd4fbfbd725dbd3de6433","created_at":"2026-10-02T14:14:01.723Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ffe2cbe6ccd542f10473e37a6b2f6149","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.729Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0c6bd4fbfbd725dbd3de6433: done: 0.017691 sec","job_status":"done","queue_duration_s":1.131528,"scheduling_latency_s":1.131559,"redis_calls":7,"redis_duration_s":0.001419,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000991,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017691,"completed_at":"2026-10-02T14:14:02.879Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 285.114091] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"aa3b84ef1f590c6f671cd067","created_at":"2026-10-02T14:14:01.754Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a325f29c45ecee49bad857b7dc9c1f4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.760Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-aa3b84ef1f590c6f671cd067: start","job_status":"start","queue_duration_s":1.120779,"scheduling_latency_s":1.120805}1431gitlab # [ 285.128875] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"aa3b84ef1f590c6f671cd067","created_at":"2026-10-02T14:14:01.754Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a325f29c45ecee49bad857b7dc9c1f4","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.760Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-aa3b84ef1f590c6f671cd067: done: 0.020478 sec","job_status":"done","queue_duration_s":1.120779,"scheduling_latency_s":1.120805,"redis_calls":7,"redis_duration_s":0.001429,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020478,"completed_at":"2026-10-02T14:14:02.901Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0016,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 285.170799] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3c59d4abb23bea63c2fc4e3a","created_at":"2026-10-02T14:14:01.776Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4886a65fe8bcd80f3d00c6ba8621e30a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:14:01.782Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3c59d4abb23bea63c2fc4e3a: start","job_status":"start","queue_duration_s":1.120591,"scheduling_latency_s":1.120619}1433gitlab # [ 285.186071] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.913Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1434gitlab # [ 285.189523] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.920Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3c59d4abb23bea63c2fc4e3a","created_at":"2026-10-02T14:14:01.776Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4886a65fe8bcd80f3d00c6ba8621e30a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:14:01.782Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3c59d4abb23bea63c2fc4e3a: done: 0.017442 sec","job_status":"done","queue_duration_s":1.120591,"scheduling_latency_s":1.120619,"redis_calls":7,"redis_duration_s":0.001315,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000362,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017442,"completed_at":"2026-10-02T14:14:02.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 285.230788] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d6ca3e2238703dcee7513b45","created_at":"2026-10-02T14:14:01.811Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d4fdff0ea232fbcee7728057eb47aba","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.839Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d6ca3e2238703dcee7513b45: start","job_status":"start","queue_duration_s":1.08327,"scheduling_latency_s":1.083295}1436gitlab # [ 285.246124] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"d6ca3e2238703dcee7513b45","created_at":"2026-10-02T14:14:01.811Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d4fdff0ea232fbcee7728057eb47aba","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.839Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-d6ca3e2238703dcee7513b45: done: 0.035557 sec","job_status":"done","queue_duration_s":1.08327,"scheduling_latency_s":1.083295,"redis_calls":10,"redis_duration_s":0.001924,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035404,"duration_s":0.035557,"completed_at":"2026-10-02T14:14:02.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004854,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 285.290560] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:02.959Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"60f0f1740a1e2b6eac98adc5","created_at":"2026-10-02T14:14:01.861Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52867824999ecf926bac0e142dd65c10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.878Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-60f0f1740a1e2b6eac98adc5: start","job_status":"start","queue_duration_s":1.08118,"scheduling_latency_s":1.081208}1438gitlab # [ 285.305526] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"60f0f1740a1e2b6eac98adc5","created_at":"2026-10-02T14:14:01.861Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52867824999ecf926bac0e142dd65c10","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.878Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-60f0f1740a1e2b6eac98adc5: done: 0.243078 sec","job_status":"done","queue_duration_s":1.08118,"scheduling_latency_s":1.081208,"redis_calls":11,"redis_duration_s":0.0022459999999999997,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000893,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.200794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006058,"exclusive_lock_hold_duration_s":0.22002206,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.243078,"completed_at":"2026-10-02T14:14:03.202Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050058,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 285.352331] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"285d66f01cdfc05b00902323","created_at":"2026-10-02T14:14:01.924Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c68cbd0af37ac18093e62b45690f6d53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:14:01.930Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-285d66f01cdfc05b00902323: start","job_status":"start","queue_duration_s":1.273759,"scheduling_latency_s":1.273785}1440gitlab # [ 285.366049] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"285d66f01cdfc05b00902323","created_at":"2026-10-02T14:14:01.924Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c68cbd0af37ac18093e62b45690f6d53","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:14:01.930Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-285d66f01cdfc05b00902323: done: 0.025694 sec","job_status":"done","queue_duration_s":1.273759,"scheduling_latency_s":1.273785,"redis_calls":7,"redis_duration_s":0.001586,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025694,"completed_at":"2026-10-02T14:14:03.229Z","load_balancing_strategy":"primary","db_duration_s":0.001998,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 285.408654] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.232Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"8881277ee8f6973dd871eaf7","created_at":"2026-10-02T14:14:01.957Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4e0da27f6403be2390721885ec1fdc9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.974Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-8881277ee8f6973dd871eaf7: start","job_status":"start","queue_duration_s":1.258092,"scheduling_latency_s":1.258117}1442gitlab # [ 285.422458] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.257Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"8881277ee8f6973dd871eaf7","created_at":"2026-10-02T14:14:01.957Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f4e0da27f6403be2390721885ec1fdc9","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.974Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-8881277ee8f6973dd871eaf7: done: 0.024823 sec","job_status":"done","queue_duration_s":1.258092,"scheduling_latency_s":1.258117,"redis_calls":12,"redis_duration_s":0.002215,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.00053,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028626,"duration_s":0.024823,"completed_at":"2026-10-02T14:14:03.256Z","load_balancing_strategy":"primary","db_duration_s":0.000724,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 285.466944] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.258Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"25bd2bef371abb28b9e6d3c0","created_at":"2026-10-02T14:14:01.979Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84574113122947b4d4fcee4f7e4f5782","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.985Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-25bd2bef371abb28b9e6d3c0: start","job_status":"start","queue_duration_s":1.272868,"scheduling_latency_s":1.272892}1444gitlab # [ 285.482269] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"25bd2bef371abb28b9e6d3c0","created_at":"2026-10-02T14:14:01.979Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"84574113122947b4d4fcee4f7e4f5782","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.985Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-25bd2bef371abb28b9e6d3c0: done: 0.020885 sec","job_status":"done","queue_duration_s":1.272868,"scheduling_latency_s":1.272892,"redis_calls":9,"redis_duration_s":0.001719,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000809,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060608,"exclusive_lock_hold_duration_s":0.0035303,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.020885,"completed_at":"2026-10-02T14:14:03.279Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001538,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 285.525890] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0bab42178989bc24a0a09dd7","created_at":"2026-10-02T14:14:01.992Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b86a91e61b04ab0f5ba8b0dc1d951837","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.999Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0bab42178989bc24a0a09dd7: start","job_status":"start","queue_duration_s":1.281549,"scheduling_latency_s":1.281573}1446gitlab # [ 285.717145] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.624Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"0bab42178989bc24a0a09dd7","created_at":"2026-10-02T14:14:01.992Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b86a91e61b04ab0f5ba8b0dc1d951837","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:01.999Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-0bab42178989bc24a0a09dd7: done: 0.342824 sec","job_status":"done","queue_duration_s":1.281549,"scheduling_latency_s":1.281573,"redis_calls":562,"redis_duration_s":0.073304,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000925,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.072379,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.310727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.342824,"completed_at":"2026-10-02T14:14:03.623Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 285.784321] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.625Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"14db85c9b30acd73d1159d62","created_at":"2026-10-02T14:14:02.023Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d6eec3b6306fbff3a2e9fefcf939b07","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.030Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-14db85c9b30acd73d1159d62: start","job_status":"start","queue_duration_s":1.595519,"scheduling_latency_s":1.595549}1448gitlab # [ 285.803773] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"14db85c9b30acd73d1159d62","created_at":"2026-10-02T14:14:02.023Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0d6eec3b6306fbff3a2e9fefcf939b07","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.030Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-14db85c9b30acd73d1159d62: done: 0.025588 sec","job_status":"done","queue_duration_s":1.595519,"scheduling_latency_s":1.595549,"redis_calls":10,"redis_duration_s":0.002379,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001354,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00682106,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.025588,"completed_at":"2026-10-02T14:14:03.651Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003161,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1449gitlab # [ 285.855709] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"35c2cffdd47fe51fa0b68ac0","created_at":"2026-10-02T14:14:02.039Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c1c75d6ac4cb7df0688663e4339e66e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.048Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-35c2cffdd47fe51fa0b68ac0: start","job_status":"start","queue_duration_s":1.604295,"scheduling_latency_s":1.604319}1450gitlab # [ 285.872794] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.668Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"35c2cffdd47fe51fa0b68ac0","created_at":"2026-10-02T14:14:02.039Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-0344a64f294b4096","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c1c75d6ac4cb7df0688663e4339e66e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.048Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-35c2cffdd47fe51fa0b68ac0: done: 0.015231 sec","job_status":"done","queue_duration_s":1.604295,"scheduling_latency_s":1.604319,"redis_calls":7,"redis_duration_s":0.001388,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000398,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015231,"completed_at":"2026-10-02T14:14:03.668Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 285.919377] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd9cbe4fc37c955f5405bd6","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd9cbe4fc37c955f5405bd6: start","job_status":"start","queue_duration_s":1.349295,"scheduling_latency_s":1.349318}1452gitlab # [ 285.933674] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.686Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8fd9cbe4fc37c955f5405bd6","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8fd9cbe4fc37c955f5405bd6: done: 0.017066 sec","job_status":"done","queue_duration_s":1.349295,"scheduling_latency_s":1.349318,"redis_calls":13,"redis_duration_s":0.002236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001422,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017066,"completed_at":"2026-10-02T14:14:03.686Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 285.978034] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa2b3c447250c613a9ffa77e","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa2b3c447250c613a9ffa77e: start","job_status":"start","queue_duration_s":1.367894,"scheduling_latency_s":1.367917}1454gitlab # [ 285.992151] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa2b3c447250c613a9ffa77e","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa2b3c447250c613a9ffa77e: done: 0.016754 sec","job_status":"done","queue_duration_s":1.367894,"scheduling_latency_s":1.367917,"redis_calls":13,"redis_duration_s":0.002167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001412,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016754,"completed_at":"2026-10-02T14:14:03.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 286.036472] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2dae456b815312a9bbf77c9","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2dae456b815312a9bbf77c9: start","job_status":"start","queue_duration_s":1.386104,"scheduling_latency_s":1.386127}1456gitlab # [ 286.049913] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2dae456b815312a9bbf77c9","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2dae456b815312a9bbf77c9: done: 0.017079 sec","job_status":"done","queue_duration_s":1.386104,"scheduling_latency_s":1.386127,"redis_calls":13,"redis_duration_s":0.002581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017079,"completed_at":"2026-10-02T14:14:03.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 286.092305] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6615646d13764d96ce5d7bd2","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6615646d13764d96ce5d7bd2: start","job_status":"start","queue_duration_s":1.404765,"scheduling_latency_s":1.404789}1458gitlab # [ 286.105286] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6615646d13764d96ce5d7bd2","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6615646d13764d96ce5d7bd2: done: 0.017015 sec","job_status":"done","queue_duration_s":1.404765,"scheduling_latency_s":1.404789,"redis_calls":13,"redis_duration_s":0.002282,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001489,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017015,"completed_at":"2026-10-02T14:14:03.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 286.146525] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6d173ad9bd987cc059b314d","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6d173ad9bd987cc059b314d: start","job_status":"start","queue_duration_s":1.42328,"scheduling_latency_s":1.423304}1460gitlab # [ 286.159675] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6d173ad9bd987cc059b314d","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6d173ad9bd987cc059b314d: done: 0.017255 sec","job_status":"done","queue_duration_s":1.42328,"scheduling_latency_s":1.423304,"redis_calls":13,"redis_duration_s":0.002264,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001473,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017255,"completed_at":"2026-10-02T14:14:03.760Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 286.202121] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de54de58f2889d97f2b5d59d","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de54de58f2889d97f2b5d59d: start","job_status":"start","queue_duration_s":1.442009,"scheduling_latency_s":1.442033}1462gitlab # [ 286.215722] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"de54de58f2889d97f2b5d59d","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-de54de58f2889d97f2b5d59d: done: 0.017087 sec","job_status":"done","queue_duration_s":1.442009,"scheduling_latency_s":1.442033,"redis_calls":13,"redis_duration_s":0.002258,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000806,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017087,"completed_at":"2026-10-02T14:14:03.779Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 286.259496] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21ae267d30b770a961b021cb","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21ae267d30b770a961b021cb: start","job_status":"start","queue_duration_s":1.46076,"scheduling_latency_s":1.460784}1464gitlab # [ 286.273513] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21ae267d30b770a961b021cb","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21ae267d30b770a961b021cb: done: 0.01716 sec","job_status":"done","queue_duration_s":1.46076,"scheduling_latency_s":1.460784,"redis_calls":13,"redis_duration_s":0.00221,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01716,"completed_at":"2026-10-02T14:14:03.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 286.317210] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f11f493b71ddbc3d4d3be0","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f11f493b71ddbc3d4d3be0: start","job_status":"start","queue_duration_s":1.479462,"scheduling_latency_s":1.479487}1466gitlab # [ 286.331010] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f11f493b71ddbc3d4d3be0","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f11f493b71ddbc3d4d3be0: done: 0.017991 sec","job_status":"done","queue_duration_s":1.479462,"scheduling_latency_s":1.479487,"redis_calls":13,"redis_duration_s":0.002537,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001711,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011614,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017991,"completed_at":"2026-10-02T14:14:03.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 286.375023] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5b136674c8113539214fb6d","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5b136674c8113539214fb6d: start","job_status":"start","queue_duration_s":1.499004,"scheduling_latency_s":1.499031}1468gitlab # [ 286.388986] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5b136674c8113539214fb6d","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5b136674c8113539214fb6d: done: 0.017799 sec","job_status":"done","queue_duration_s":1.499004,"scheduling_latency_s":1.499031,"redis_calls":13,"redis_duration_s":0.00243,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017799,"completed_at":"2026-10-02T14:14:03.837Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 286.433929] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e3770f535904ee25a37522","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e3770f535904ee25a37522: start","job_status":"start","queue_duration_s":1.518319,"scheduling_latency_s":1.518344}1470gitlab # [ 286.447463] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e3770f535904ee25a37522","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e3770f535904ee25a37522: done: 0.01749 sec","job_status":"done","queue_duration_s":1.518319,"scheduling_latency_s":1.518344,"redis_calls":13,"redis_duration_s":0.0023420000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001528,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01749,"completed_at":"2026-10-02T14:14:03.856Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 286.488857] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"898a544c4994bead9f6aac5c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-898a544c4994bead9f6aac5c: start","job_status":"start","queue_duration_s":1.537358,"scheduling_latency_s":1.537383}1472gitlab # [ 286.501781] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"898a544c4994bead9f6aac5c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-898a544c4994bead9f6aac5c: done: 0.017199 sec","job_status":"done","queue_duration_s":1.537358,"scheduling_latency_s":1.537383,"redis_calls":13,"redis_duration_s":0.002212,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00081,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001402,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01131,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017199,"completed_at":"2026-10-02T14:14:03.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 286.542396] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.876Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9def72ae73e1c6752d16ca7b","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9def72ae73e1c6752d16ca7b: start","job_status":"start","queue_duration_s":1.556168,"scheduling_latency_s":1.556192}1474gitlab # [ 286.555954] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9def72ae73e1c6752d16ca7b","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9def72ae73e1c6752d16ca7b: done: 0.01766 sec","job_status":"done","queue_duration_s":1.556168,"scheduling_latency_s":1.556192,"redis_calls":13,"redis_duration_s":0.002225,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01766,"completed_at":"2026-10-02T14:14:03.894Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 286.599619] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7168a05cbe2b2aecd897a806","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7168a05cbe2b2aecd897a806: start","job_status":"start","queue_duration_s":1.57572,"scheduling_latency_s":1.575747}1476gitlab # [ 286.613719] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7168a05cbe2b2aecd897a806","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7168a05cbe2b2aecd897a806: done: 0.017784 sec","job_status":"done","queue_duration_s":1.57572,"scheduling_latency_s":1.575747,"redis_calls":13,"redis_duration_s":0.002393,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001567,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017784,"completed_at":"2026-10-02T14:14:03.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 286.658405] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"335df1bc2f839a3a3038a389","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-335df1bc2f839a3a3038a389: start","job_status":"start","queue_duration_s":1.59496,"scheduling_latency_s":1.59498}1478gitlab # [ 286.672272] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"335df1bc2f839a3a3038a389","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-335df1bc2f839a3a3038a389: done: 0.0154 sec","job_status":"done","queue_duration_s":1.59496,"scheduling_latency_s":1.59498,"redis_calls":13,"redis_duration_s":0.002016,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0154,"completed_at":"2026-10-02T14:14:03.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 286.715598] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edb2f55396e74477b11209fb","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edb2f55396e74477b11209fb: start","job_status":"start","queue_duration_s":1.611573,"scheduling_latency_s":1.611593}1480gitlab # [ 286.729551] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edb2f55396e74477b11209fb","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edb2f55396e74477b11209fb: done: 0.014808 sec","job_status":"done","queue_duration_s":1.611573,"scheduling_latency_s":1.611593,"redis_calls":13,"redis_duration_s":0.001885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00069,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014808,"completed_at":"2026-10-02T14:14:03.946Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 286.773354] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.947Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"756d7a840c44d1874e3a6b11","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-756d7a840c44d1874e3a6b11: start","job_status":"start","queue_duration_s":1.627603,"scheduling_latency_s":1.627622}1482gitlab # [ 286.787307] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"756d7a840c44d1874e3a6b11","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-756d7a840c44d1874e3a6b11: done: 0.014939 sec","job_status":"done","queue_duration_s":1.627603,"scheduling_latency_s":1.627622,"redis_calls":13,"redis_duration_s":0.00196,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014939,"completed_at":"2026-10-02T14:14:03.962Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 286.830927] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.964Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa7561b9df38950577f18186","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa7561b9df38950577f18186: start","job_status":"start","queue_duration_s":1.643871,"scheduling_latency_s":1.643891}1484gitlab # [ 286.845887] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa7561b9df38950577f18186","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa7561b9df38950577f18186: done: 0.014724 sec","job_status":"done","queue_duration_s":1.643871,"scheduling_latency_s":1.643891,"redis_calls":13,"redis_duration_s":0.001856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014724,"completed_at":"2026-10-02T14:14:03.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 286.889684] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09058847a8c85440b8ae1b0c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09058847a8c85440b8ae1b0c: start","job_status":"start","queue_duration_s":1.659827,"scheduling_latency_s":1.659846}1486gitlab # [ 286.903595] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09058847a8c85440b8ae1b0c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09058847a8c85440b8ae1b0c: done: 0.014813 sec","job_status":"done","queue_duration_s":1.659827,"scheduling_latency_s":1.659846,"redis_calls":13,"redis_duration_s":0.001934,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001242,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014813,"completed_at":"2026-10-02T14:14:03.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 286.947477] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:03.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f468495dcb7892db34c5d6b","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f468495dcb7892db34c5d6b: start","job_status":"start","queue_duration_s":1.675856,"scheduling_latency_s":1.675876}1488gitlab # [ 286.961313] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f468495dcb7892db34c5d6b","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f468495dcb7892db34c5d6b: done: 0.014834 sec","job_status":"done","queue_duration_s":1.675856,"scheduling_latency_s":1.675876,"redis_calls":13,"redis_duration_s":0.001903,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014834,"completed_at":"2026-10-02T14:14:04.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 287.002353] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9340d1029216a31024adba78","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9340d1029216a31024adba78: start","job_status":"start","queue_duration_s":1.691879,"scheduling_latency_s":1.691898}1490gitlab # [ 287.015727] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9340d1029216a31024adba78","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9340d1029216a31024adba78: done: 0.015052 sec","job_status":"done","queue_duration_s":1.691879,"scheduling_latency_s":1.691898,"redis_calls":13,"redis_duration_s":0.001979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001295,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015052,"completed_at":"2026-10-02T14:14:04.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 287.058223] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e74e81fff0a6df1d5f4f0f1","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e74e81fff0a6df1d5f4f0f1: start","job_status":"start","queue_duration_s":1.708145,"scheduling_latency_s":1.708164}1492gitlab # [ 287.071413] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.043Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e74e81fff0a6df1d5f4f0f1","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e74e81fff0a6df1d5f4f0f1: done: 0.015416 sec","job_status":"done","queue_duration_s":1.708145,"scheduling_latency_s":1.708164,"redis_calls":13,"redis_duration_s":0.002003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000687,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001316,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015416,"completed_at":"2026-10-02T14:14:04.043Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 287.113773] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"905234375bc3af2d2a26029c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-905234375bc3af2d2a26029c: start","job_status":"start","queue_duration_s":1.724845,"scheduling_latency_s":1.724863}1494gitlab # [ 287.127145] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.060Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"905234375bc3af2d2a26029c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-905234375bc3af2d2a26029c: done: 0.015754 sec","job_status":"done","queue_duration_s":1.724845,"scheduling_latency_s":1.724863,"redis_calls":13,"redis_duration_s":0.002034,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001334,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015754,"completed_at":"2026-10-02T14:14:04.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 287.169818] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6885194bcd85a1f034ba67f7","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6885194bcd85a1f034ba67f7: start","job_status":"start","queue_duration_s":1.741872,"scheduling_latency_s":1.741891}1496gitlab # [ 287.183146] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6885194bcd85a1f034ba67f7","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6885194bcd85a1f034ba67f7: done: 0.015669 sec","job_status":"done","queue_duration_s":1.741872,"scheduling_latency_s":1.741891,"redis_calls":13,"redis_duration_s":0.001977,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000712,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010522,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015669,"completed_at":"2026-10-02T14:14:04.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 287.225602] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dadafebcfd32b6e370258a32","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dadafebcfd32b6e370258a32: start","job_status":"start","queue_duration_s":1.758845,"scheduling_latency_s":1.758865}1498gitlab # [ 287.238928] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dadafebcfd32b6e370258a32","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dadafebcfd32b6e370258a32: done: 0.016065 sec","job_status":"done","queue_duration_s":1.758845,"scheduling_latency_s":1.758865,"redis_calls":13,"redis_duration_s":0.00221,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001442,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016065,"completed_at":"2026-10-02T14:14:04.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 287.282057] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09f4027ff6b7338d649e373c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09f4027ff6b7338d649e373c: start","job_status":"start","queue_duration_s":1.7763,"scheduling_latency_s":1.776321}1500gitlab # [ 287.295757] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"09f4027ff6b7338d649e373c","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-09f4027ff6b7338d649e373c: done: 0.016344 sec","job_status":"done","queue_duration_s":1.7763,"scheduling_latency_s":1.776321,"redis_calls":13,"redis_duration_s":0.002188,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001433,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010844,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016344,"completed_at":"2026-10-02T14:14:04.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 287.339148] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.114Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a925850309f6e97a64d2ce26","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a925850309f6e97a64d2ce26: start","job_status":"start","queue_duration_s":1.793945,"scheduling_latency_s":1.793967}1502gitlab # [ 287.353004] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a925850309f6e97a64d2ce26","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a925850309f6e97a64d2ce26: done: 0.018068 sec","job_status":"done","queue_duration_s":1.793945,"scheduling_latency_s":1.793967,"redis_calls":13,"redis_duration_s":0.00236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001425,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018068,"completed_at":"2026-10-02T14:14:04.132Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 287.394555] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.134Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e4ed36692ab220c8041e87","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e4ed36692ab220c8041e87: start","job_status":"start","queue_duration_s":1.814105,"scheduling_latency_s":1.814136}1504gitlab # [ 287.407827] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e4ed36692ab220c8041e87","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e4ed36692ab220c8041e87: done: 0.019701 sec","job_status":"done","queue_duration_s":1.814105,"scheduling_latency_s":1.814136,"redis_calls":13,"redis_duration_s":0.0026579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001104,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001554,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019701,"completed_at":"2026-10-02T14:14:04.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 287.450967] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d94df813fcfdc039691360a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d94df813fcfdc039691360a: start","job_status":"start","queue_duration_s":1.835197,"scheduling_latency_s":1.835254}1506gitlab # [ 287.464292] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d94df813fcfdc039691360a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d94df813fcfdc039691360a: done: 0.016852 sec","job_status":"done","queue_duration_s":1.835197,"scheduling_latency_s":1.835254,"redis_calls":13,"redis_duration_s":0.002295,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001509,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016852,"completed_at":"2026-10-02T14:14:04.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 287.506142] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cf914edf9f9354008d14a78","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cf914edf9f9354008d14a78: start","job_status":"start","queue_duration_s":1.853412,"scheduling_latency_s":1.853433}1508gitlab # [ 287.520064] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6cf914edf9f9354008d14a78","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6cf914edf9f9354008d14a78: done: 0.016807 sec","job_status":"done","queue_duration_s":1.853412,"scheduling_latency_s":1.853433,"redis_calls":13,"redis_duration_s":0.0022329999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016807,"completed_at":"2026-10-02T14:14:04.190Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 287.564258] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.192Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd28fb8615d4098a0c736449","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd28fb8615d4098a0c736449: start","job_status":"start","queue_duration_s":1.871685,"scheduling_latency_s":1.871708}1510gitlab # [ 287.578091] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd28fb8615d4098a0c736449","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd28fb8615d4098a0c736449: done: 0.017838 sec","job_status":"done","queue_duration_s":1.871685,"scheduling_latency_s":1.871708,"redis_calls":13,"redis_duration_s":0.00234,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000827,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001513,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017838,"completed_at":"2026-10-02T14:14:04.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 287.620326] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.211Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8f2e5b689450013ad3ceb33","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8f2e5b689450013ad3ceb33: start","job_status":"start","queue_duration_s":1.890994,"scheduling_latency_s":1.891019}1512gitlab # [ 287.633353] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8f2e5b689450013ad3ceb33","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8f2e5b689450013ad3ceb33: done: 0.01923 sec","job_status":"done","queue_duration_s":1.890994,"scheduling_latency_s":1.891019,"redis_calls":13,"redis_duration_s":0.002745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01923,"completed_at":"2026-10-02T14:14:04.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 287.675372] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"589463ddf39036ecbf520eef","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-589463ddf39036ecbf520eef: start","job_status":"start","queue_duration_s":1.911941,"scheduling_latency_s":1.911968}1514gitlab # [ 287.688582] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"589463ddf39036ecbf520eef","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-589463ddf39036ecbf520eef: done: 0.019221 sec","job_status":"done","queue_duration_s":1.911941,"scheduling_latency_s":1.911968,"redis_calls":13,"redis_duration_s":0.002406,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000928,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001478,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019221,"completed_at":"2026-10-02T14:14:04.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 287.729413] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"313541a11d9eba0b1c60071a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-313541a11d9eba0b1c60071a: start","job_status":"start","queue_duration_s":1.932848,"scheduling_latency_s":1.932877}1516gitlab # [ 287.742448] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.273Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"313541a11d9eba0b1c60071a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-313541a11d9eba0b1c60071a: done: 0.020021 sec","job_status":"done","queue_duration_s":1.932848,"scheduling_latency_s":1.932877,"redis_calls":13,"redis_duration_s":0.002371,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001466,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020021,"completed_at":"2026-10-02T14:14:04.273Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 287.783529] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6079c0f66fbdb08828ed6a8","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6079c0f66fbdb08828ed6a8: start","job_status":"start","queue_duration_s":1.954573,"scheduling_latency_s":1.954602}1518gitlab # [ 287.796648] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.295Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6079c0f66fbdb08828ed6a8","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6079c0f66fbdb08828ed6a8: done: 0.020971 sec","job_status":"done","queue_duration_s":1.954573,"scheduling_latency_s":1.954602,"redis_calls":13,"redis_duration_s":0.0026579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001637,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020971,"completed_at":"2026-10-02T14:14:04.295Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 287.837912] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25a62f67d5f7ce23e9f24375","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25a62f67d5f7ce23e9f24375: start","job_status":"start","queue_duration_s":1.977259,"scheduling_latency_s":1.977291}1520gitlab # [ 287.851014] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25a62f67d5f7ce23e9f24375","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25a62f67d5f7ce23e9f24375: done: 0.02098 sec","job_status":"done","queue_duration_s":1.977259,"scheduling_latency_s":1.977291,"redis_calls":13,"redis_duration_s":0.002503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001603,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02098,"completed_at":"2026-10-02T14:14:04.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 287.893887] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbc6ac0f738dd4c45f834b39","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbc6ac0f738dd4c45f834b39: start","job_status":"start","queue_duration_s":2.000017,"scheduling_latency_s":2.00005}1522gitlab # [ 287.907212] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbc6ac0f738dd4c45f834b39","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbc6ac0f738dd4c45f834b39: done: 0.022664 sec","job_status":"done","queue_duration_s":2.000017,"scheduling_latency_s":2.00005,"redis_calls":13,"redis_duration_s":0.002665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000981,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001684,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022664,"completed_at":"2026-10-02T14:14:04.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 287.949522] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be4bd6fcb3d1a9e72547ca9e","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be4bd6fcb3d1a9e72547ca9e: start","job_status":"start","queue_duration_s":2.024679,"scheduling_latency_s":2.024719}1524gitlab # [ 287.962791] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be4bd6fcb3d1a9e72547ca9e","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be4bd6fcb3d1a9e72547ca9e: done: 0.018923 sec","job_status":"done","queue_duration_s":2.024679,"scheduling_latency_s":2.024719,"redis_calls":13,"redis_duration_s":0.002502,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001539,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018923,"completed_at":"2026-10-02T14:14:04.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 288.004914] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.365Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"547a447adac37c32f8e99978","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-547a447adac37c32f8e99978: start","job_status":"start","queue_duration_s":2.045039,"scheduling_latency_s":2.04506}1526gitlab # [ 288.018050] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"547a447adac37c32f8e99978","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-547a447adac37c32f8e99978: done: 0.018789 sec","job_status":"done","queue_duration_s":2.045039,"scheduling_latency_s":2.04506,"redis_calls":13,"redis_duration_s":0.003925,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003128,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018789,"completed_at":"2026-10-02T14:14:04.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 288.060093] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2d37f3b01bbecc726ac9ce","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2d37f3b01bbecc726ac9ce: start","job_status":"start","queue_duration_s":2.065296,"scheduling_latency_s":2.065316}1528gitlab # [ 288.074378] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff2d37f3b01bbecc726ac9ce","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff2d37f3b01bbecc726ac9ce: done: 0.01684 sec","job_status":"done","queue_duration_s":2.065296,"scheduling_latency_s":2.065316,"redis_calls":13,"redis_duration_s":0.001967,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01684,"completed_at":"2026-10-02T14:14:04.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 288.115890] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be95865bc11a2b70671fb8ff","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be95865bc11a2b70671fb8ff: start","job_status":"start","queue_duration_s":2.083614,"scheduling_latency_s":2.083637}1530gitlab # [ 288.129160] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be95865bc11a2b70671fb8ff","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be95865bc11a2b70671fb8ff: done: 0.017028 sec","job_status":"done","queue_duration_s":2.083614,"scheduling_latency_s":2.083637,"redis_calls":13,"redis_duration_s":0.002004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017028,"completed_at":"2026-10-02T14:14:04.420Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 288.170336] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e032cef3f0c97afd2e0a6289","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e032cef3f0c97afd2e0a6289: start","job_status":"start","queue_duration_s":2.102108,"scheduling_latency_s":2.102129}1532gitlab # [ 288.183469] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e032cef3f0c97afd2e0a6289","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e032cef3f0c97afd2e0a6289: done: 0.017657 sec","job_status":"done","queue_duration_s":2.102108,"scheduling_latency_s":2.102129,"redis_calls":13,"redis_duration_s":0.002173,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000729,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001444,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017657,"completed_at":"2026-10-02T14:14:04.440Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 288.225016] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89e44f52681c8611d7dd5d67","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89e44f52681c8611d7dd5d67: start","job_status":"start","queue_duration_s":2.121561,"scheduling_latency_s":2.121585}1534gitlab # [ 288.238129] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89e44f52681c8611d7dd5d67","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89e44f52681c8611d7dd5d67: done: 0.018899 sec","job_status":"done","queue_duration_s":2.121561,"scheduling_latency_s":2.121585,"redis_calls":13,"redis_duration_s":0.00217,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001407,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018899,"completed_at":"2026-10-02T14:14:04.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 288.280582] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"764acf38380a4f868cd9dd0a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-764acf38380a4f868cd9dd0a: start","job_status":"start","queue_duration_s":2.142286,"scheduling_latency_s":2.142313}1536gitlab # [ 288.301943] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"764acf38380a4f868cd9dd0a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-764acf38380a4f868cd9dd0a: done: 0.022603 sec","job_status":"done","queue_duration_s":2.142286,"scheduling_latency_s":2.142313,"redis_calls":13,"redis_duration_s":0.002653,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000929,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001724,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022603,"completed_at":"2026-10-02T14:14:04.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 288.363873] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750a4349ddca01d2832201aa","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750a4349ddca01d2832201aa: start","job_status":"start","queue_duration_s":2.166907,"scheduling_latency_s":2.166936}1538gitlab # [ 288.379455] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.511Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750a4349ddca01d2832201aa","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750a4349ddca01d2832201aa: done: 0.024526 sec","job_status":"done","queue_duration_s":2.166907,"scheduling_latency_s":2.166936,"redis_calls":13,"redis_duration_s":0.0028710000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001683,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024526,"completed_at":"2026-10-02T14:14:04.511Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 288.425270] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fce251ad3c57b2a884c1dc29","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fce251ad3c57b2a884c1dc29: start","job_status":"start","queue_duration_s":2.193433,"scheduling_latency_s":2.193464}1540gitlab # [ 288.439666] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fce251ad3c57b2a884c1dc29","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fce251ad3c57b2a884c1dc29: done: 0.024123 sec","job_status":"done","queue_duration_s":2.193433,"scheduling_latency_s":2.193464,"redis_calls":13,"redis_duration_s":0.002483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000817,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001666,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024123,"completed_at":"2026-10-02T14:14:04.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 288.483888] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8815285ec40cb17fe7cbbbd4","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8815285ec40cb17fe7cbbbd4: start","job_status":"start","queue_duration_s":2.219824,"scheduling_latency_s":2.219858}1542gitlab # [ 288.498763] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8815285ec40cb17fe7cbbbd4","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8815285ec40cb17fe7cbbbd4: done: 0.024614 sec","job_status":"done","queue_duration_s":2.219824,"scheduling_latency_s":2.219858,"redis_calls":13,"redis_duration_s":0.002204,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000798,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001406,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024614,"completed_at":"2026-10-02T14:14:04.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 288.542173] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed8dcf8fec9bb9facbd0dbac","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed8dcf8fec9bb9facbd0dbac: start","job_status":"start","queue_duration_s":2.246513,"scheduling_latency_s":2.246545}1544gitlab # [ 288.555931] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.591Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed8dcf8fec9bb9facbd0dbac","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed8dcf8fec9bb9facbd0dbac: done: 0.024371 sec","job_status":"done","queue_duration_s":2.246513,"scheduling_latency_s":2.246545,"redis_calls":13,"redis_duration_s":0.002366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001477,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024371,"completed_at":"2026-10-02T14:14:04.591Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 288.599382] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"183248212a99c1c38aa57d93","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-183248212a99c1c38aa57d93: start","job_status":"start","queue_duration_s":2.273163,"scheduling_latency_s":2.273195}1546gitlab # [ 288.613550] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.618Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"183248212a99c1c38aa57d93","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-183248212a99c1c38aa57d93: done: 0.024657 sec","job_status":"done","queue_duration_s":2.273163,"scheduling_latency_s":2.273195,"redis_calls":13,"redis_duration_s":0.002355,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001486,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016562,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024657,"completed_at":"2026-10-02T14:14:04.618Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 288.658058] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.620Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b850d4ec3c56004ac187013","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b850d4ec3c56004ac187013: start","job_status":"start","queue_duration_s":2.299759,"scheduling_latency_s":2.29979}1548gitlab # [ 288.672336] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b850d4ec3c56004ac187013","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b850d4ec3c56004ac187013: done: 0.025947 sec","job_status":"done","queue_duration_s":2.299759,"scheduling_latency_s":2.29979,"redis_calls":13,"redis_duration_s":0.002722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000867,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001855,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025947,"completed_at":"2026-10-02T14:14:04.645Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 288.717858] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdd4ecd160e5b81b20fb0f9a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdd4ecd160e5b81b20fb0f9a: start","job_status":"start","queue_duration_s":2.327577,"scheduling_latency_s":2.327611}1550gitlab # [ 288.732292] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:04.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdd4ecd160e5b81b20fb0f9a","created_at":"2026-10-02T14:14:02.187Z","trace_propagation_headers":{"sentry-trace":"ab65a74f42be4325b843cfe5d1d9a337-c6d3611d90714951","baggage":"sentry-trace_id=ab65a74f42be4325b843cfe5d1d9a337,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7ed671c5ba75fb93cdd82ec6e5b4c54b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:02.320Z","job_size_bytes":2,"pid":1955,"sidekiq_tid":"maqn","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdd4ecd160e5b81b20fb0f9a: done: 0.025248 sec","job_status":"done","queue_duration_s":2.327577,"scheduling_latency_s":2.327611,"redis_calls":13,"redis_duration_s":0.00266,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001787,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025248,"completed_at":"2026-10-02T14:14:04.672Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 305.866344] postgres[3296]: [3296] ERROR: relation "table_without_models" does not exist1552gitlab # [ 305.869037] postgres[3296]: [3296] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1553gitlab # [ 308.150182] redis[798]: 10 changes in 300 seconds. Saving...1554gitlab # [ 308.150570] redis[798]: Background saving started by pid 33191555gitlab # [ 308.155838] redis[3319]: BGSAVE done, 453 keys saved, 0 keys skipped, 117578 bytes written.1556gitlab # [ 308.159928] redis[3319]: DB saved on disk1557gitlab # [ 308.161266] redis[3319]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1558gitlab # [ 308.250989] redis[798]: Background saving terminated with success1559gitlab # [ 309.782015] postgres[866]: [866] LOG: checkpoint starting: time1560gitlab # [ 311.262736] gitaly[924]: time="2026-10-02T14:14:29.169Z" level=info msg="burden monitor stats" pid=9241561gitlab # [ 311.531476] runuser[2942]: pam_unix(runuser:session): session closed for user gitlab1562gitlab # [ 311.534140] setup-gitlab-start[2905]: + systemctl restart gitlab-sidekiq.service1563gitlab # [ 311.559890] systemd[1]: Stopping gitlab-sidekiq.service...1564gitlab # [ 311.561492] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:29.467Z","message":"Shutting down"}1565gitlab # [ 311.563094] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:29.468Z","message":"Scheduler exiting..."}1566gitlab # [ 311.564820] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:29.468Z","message":"Terminating quiet threads for default capsule"}1567gitlab # [ 311.566729] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:29.468Z","message":"Scheduler exiting..."}1568gitlab # [ 312.064371] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:29.971Z","message":"Bye!"}1569gitlab # [ 312.094963] sidekiq-cluster[1955]: {"severity":"INFO","time":"2026-10-02T14:14:30.001Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1955,"worker_id":"sidekiq_0","memwd_rss_bytes":957513728,"retry":0}1570gitlab # [ 316.175193] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1571gitlab # [ 316.175500] systemd[1]: Stopped gitlab-sidekiq.service.1572gitlab # [ 316.175616] systemd[1]: gitlab-sidekiq.service: Consumed 1min 59.203s CPU time over 3min 25.881s wall clock time, 946.6M memory peak, 4.3M read from disk.1573gitlab # [ 316.183532] systemd[1]: Started gitlab-sidekiq.service.1574gitlab # [ 316.186598] setup-gitlab-start[2905]: + mkdir -p /var/lib/secrets1575gitlab # [ 316.189971] setup-gitlab-start[2905]: + echo nixbot-test-token-of-20-chars1576gitlab # [ 316.191647] setup-gitlab-start[2905]: + chmod 644 /var/lib/secrets/gitlab-token1577gitlab # [ 316.194106] setup-gitlab-start[2905]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1578gitlab # [ 316.936455] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:34 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 7121579gitlab # [ 317.715075] gitaly[924]: time="2026-10-02T14:14:35.621Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:45.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.621" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1580gitlab # [ 317.715415] gitaly[924]: time="2026-10-02T14:14:35.622Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:45.000" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.621" grpc.time_ms=0.419 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1581gitlab # [ 317.939461] gitaly[924]: time="2026-10-02T14:14:35.846Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:45.226" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.846" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1582gitlab # [ 317.957837] gitaly[924]: time="2026-10-02T14:14:35.847Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:45.226" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.846" grpc.time_ms=0.384 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 318.006470] gitaly[924]: time="2026-10-02T14:14:35.913Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR diskcache=7f65c584-391f-4928-ae35-45a2a2be2784 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:14:45.253" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.874" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 318.023973] gitaly[924]: time="2026-10-02T14:14:35.914Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1441 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:14:45.253" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.874" grpc.time_ms=39.354 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 318.045422] gitaly[924]: time="2026-10-02T14:14:35.927Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:45.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.926" pid=924 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1586gitlab # [ 318.061765] gitaly[924]: time="2026-10-02T14:14:35.928Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:45.006" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.926" grpc.time_ms=1.133 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1587gitlab # [ 318.077424] gitaly[924]: time="2026-10-02T14:14:35.942Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:14:45.012" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:35.932" grpc.time_ms=9.063 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1588gitlab # [ 318.107452] gitaly[924]: time="2026-10-02T14:14:36.014Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:46.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.014" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1589gitlab # [ 318.124489] gitaly[924]: time="2026-10-02T14:14:36.015Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:46.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.014" grpc.time_ms=0.4 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 318.146924] gitaly[924]: time="2026-10-02T14:14:36.054Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR diskcache=1590c779-611d-4bc8-9c56-9c096ec2544c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:14:46.007" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.017" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1591gitlab # [ 318.160870] gitaly[924]: time="2026-10-02T14:14:36.054Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=18 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T14:14:46.007" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.017" grpc.time_ms=36.85 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1592gitlab # [ 318.177584] gitaly[924]: time="2026-10-02T14:14:36.062Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:46.050" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.061" pid=924 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1593gitlab # [ 318.189810] gitaly[924]: time="2026-10-02T14:14:36.062Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:46.050" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.061" grpc.time_ms=0.836 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594gitlab # [ 318.202488] gitaly[924]: time="2026-10-02T14:14:36.078Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:14:46.056" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.067" grpc.time_ms=10.77 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595gitlab # [ 318.218178] gitaly[924]: time="2026-10-02T14:14:36.117Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:14:46.106" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.117" grpc.time_ms=0.229 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596gitlab # [ 318.248853] gitaly[924]: time="2026-10-02T14:14:36.156Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:14:46.145" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.155" grpc.time_ms=0.498 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1597gitlab # [ 318.852486] gitaly[924]: time="2026-10-02T14:14:36.759Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF13A00Q2V3WJRV973VVR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:14:46.128" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:36.748" grpc.time_ms=10.288 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598gitlab # [ 320.512123] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:38 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 35591599gitlab # [ 320.516508] setup-gitlab-start[3373]: {"id":1,"message":null,"error":null}1600gitlab # [ 321.522172] sidekiq-cluster[3357]: {"severity":"INFO","time":"2026-10-02T14:14:39.400Z","message":"Starting cluster with 1 processes"}1601gitlab # [ 325.806278] gitaly[924]: time="2026-10-02T14:14:43.713Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFF9FP230QZS6FG0CT7P6K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:14:53.082" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:43.702" grpc.time_ms=10.286 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602gitlab # [ 325.976849] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:43 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 4371603gitlab # [ 325.980647] setup-gitlab-start[2905]: + mkdir -p /tmp/test-flake1604gitlab # [ 325.983775] setup-gitlab-start[2905]: + cd /tmp/test-flake1605gitlab # [ 325.985677] setup-gitlab-start[2905]: + git init -b master1606gitlab # [ 325.998400] setup-gitlab-start[3453]: Initialized empty Git repository in /tmp/test-flake/.git/1607gitlab # [ 326.000424] setup-gitlab-start[2905]: + git config user.name test1608gitlab # [ 326.002248] setup-gitlab-start[2905]: + git config user.email test@example.com1609gitlab # [ 326.005860] setup-gitlab-start[2905]: + cat1610gitlab # [ 326.009076] setup-gitlab-start[2905]: + git add flake.nix1611gitlab # [ 326.013344] setup-gitlab-start[2905]: + git commit -m 'initial commit'1612gitlab # [ 326.029951] setup-gitlab-start[3458]: [master (root-commit) 88b61d9] initial commit1613gitlab # [ 326.031737] setup-gitlab-start[3458]: 1 file changed, 10 insertions(+)1614gitlab # [ 326.034015] setup-gitlab-start[3458]: create mode 100644 flake.nix1615gitlab # [ 326.035458] setup-gitlab-start[2905]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1616gitlab # [ 326.040581] setup-gitlab-start[2905]: + git push -u origin master1617gitlab # [ 326.193204] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:44 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1241618gitlab # [ 326.620288] gitaly[924]: time="2026-10-02T14:14:44.527Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFA455HT7VNRSCJVZ2RJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:44.514" grpc.time_ms=12.536 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1619gitlab # [ 326.643314] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:44 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 4261620gitlab # [ 327.252910] gitaly[924]: time="2026-10-02T14:14:45.159Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:14:55.137" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:45.148" grpc.time_ms=11.246 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1621gitlab # [ 327.350963] gitaly[924]: time="2026-10-02T14:14:45.258Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T14:15:15.046" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T14:14:45.246" grpc.time_ms=11.327 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1622gitlab # [ 327.437881] gitaly[924]: time="2026-10-02T14:14:45.345Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YFFAHW9TRM88X23S3NB33S duration_ms=446ns method=POST pid=924 status=200 url="http://unix/api/v4/internal/allowed"1623gitlab # [ 327.592364] gitaly[924]: time="2026-10-02T14:14:45.499Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YFFAHW9TRM88X23S3NB33S duration_ms=153ns method=POST pid=924 status=200 url="http://unix/api/v4/internal/pre_receive"1624gitlab # [ 327.599734] gitaly[924]: time="2026-10-02T14:14:45.500Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:14:44.897" grpc.time_ms=602.262 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1625gitlab # [ 327.648960] gitaly[924]: time="2026-10-02T14:14:45.556Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:14:45.554" grpc.time_ms=1.271 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1626gitlab # [ 327.693824] gitaly[924]: time="2026-10-02T14:14:45.601Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:14:45.600" grpc.time_ms=0.688 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1627gitlab # [ 327.746155] gitaly[924]: time="2026-10-02T14:14:45.653Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:14:45.652" grpc.time_ms=0.637 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1628gitlab # [ 328.047378] gitaly[924]: time="2026-10-02T14:14:45.954Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:14:55.023" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:45.943" grpc.time_ms=10.931 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1629gitlab # [ 328.088304] gitaly[924]: time="2026-10-02T14:14:45.995Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T14:14:55.052" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:14:45.973" grpc.time_ms=22.045 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1630gitlab # [ 328.125875] gitaly[924]: time="2026-10-02T14:14:46.033Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YFFAHW9TRM88X23S3NB33S duration_ms=328ns method=POST pid=924 status=200 url="http://unix/api/v4/internal/post_receive"1631gitlab # [ 328.134550] gitaly[924]: time="2026-10-02T14:14:46.034Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:14:45.701" grpc.time_ms=332.342 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1632gitlab # [ 328.155715] gitaly[924]: time="2026-10-02T14:14:46.063Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S diskcache=e12f122a-912b-44f2-9d3f-89e333aebeba grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:44.831" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1633gitlab # [ 328.173052] gitaly[924]: time="2026-10-02T14:14:46.063Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=212 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13267 command.oublock=32 command.real_time_ms=1216 command.spawn_token_fork_ms=0 command.system_time_ms=107 command.user_time_ms=105 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:44.831" grpc.time_ms=1232.185 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1634gitlab # [ 328.196395] gitlab-workhorse[823]: time="2026-10-02T14:14:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521635gitlab # [ 328.199438] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:46 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 15231636gitlab # [ 328.203121] systemd[1]: Finished setup-gitlab.service.1637gitlab # [ 328.204510] setup-gitlab-start[3462]: To http://gitlab/root/test-flake.git1638gitlab # [ 328.205837] setup-gitlab-start[3462]: * [new branch] master -> master1639gitlab # [ 328.207162] setup-gitlab-start[3462]: branch 'master' set up to track 'origin/master'.1640gitlab # [ 328.208722] systemd[1]: Starting nixbot CI service...1641gitlab: (finished: waiting for unit setup-gitlab.service, in 329.36 seconds)1642gitlab: waiting for unit nixbot.service1643gitlab # [ 329.329517] nixbot[3547]: {"ts": "2026-10-02T14:14:47.236Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1644gitlab # [ 329.361533] nixbot[3547]: {"ts": "2026-10-02T14:14:47.268Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1645gitlab # [ 329.385951] nixbot[3547]: {"ts": "2026-10-02T14:14:47.293Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1646gitlab # [ 329.390691] nixbot[3547]: {"ts": "2026-10-02T14:14:47.298Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1647gitlab # [ 329.394852] nixbot[3547]: {"ts": "2026-10-02T14:14:47.302Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1648gitlab # [ 329.401790] nixbot[3547]: {"ts": "2026-10-02T14:14:47.309Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1649gitlab # [ 329.406268] nixbot[3547]: {"ts": "2026-10-02T14:14:47.313Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1650gitlab # [ 329.409747] nixbot[3547]: {"ts": "2026-10-02T14:14:47.315Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1651gitlab # [ 329.413942] nixbot[3547]: {"ts": "2026-10-02T14:14:47.321Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1652gitlab # [ 329.417414] nixbot[3547]: {"ts": "2026-10-02T14:14:47.322Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1653gitlab # [ 329.420948] nixbot[3547]: {"ts": "2026-10-02T14:14:47.328Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1654gitlab # [ 329.427293] nixbot[3547]: {"ts": "2026-10-02T14:14:47.334Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1655gitlab # [ 329.430857] nixbot[3547]: {"ts": "2026-10-02T14:14:47.338Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1656gitlab # [ 329.434476] nixbot[3547]: {"ts": "2026-10-02T14:14:47.339Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1657gitlab # [ 329.438204] nixbot[3547]: {"ts": "2026-10-02T14:14:47.343Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1658gitlab # [ 329.441881] nixbot[3547]: {"ts": "2026-10-02T14:14:47.345Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1659gitlab # [ 329.445314] nixbot[3547]: {"ts": "2026-10-02T14:14:47.346Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1660gitlab # [ 329.448927] nixbot[3547]: {"ts": "2026-10-02T14:14:47.347Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1661gitlab # [ 329.452240] nixbot[3547]: {"ts": "2026-10-02T14:14:47.352Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1662gitlab # [ 329.455638] nixbot[3547]: {"ts": "2026-10-02T14:14:47.353Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1663gitlab # [ 329.459354] nixbot[3547]: {"ts": "2026-10-02T14:14:47.354Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1664gitlab # [ 329.462936] nixbot[3547]: {"ts": "2026-10-02T14:14:47.359Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1665gitlab # [ 329.466437] nixbot[3547]: {"ts": "2026-10-02T14:14:47.360Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1666gitlab # [ 329.469813] nixbot[3547]: {"ts": "2026-10-02T14:14:47.362Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1667gitlab # [ 329.472994] nixbot[3547]: {"ts": "2026-10-02T14:14:47.363Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1668gitlab # [ 329.476057] nixbot[3547]: {"ts": "2026-10-02T14:14:47.365Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1669gitlab # [ 329.479228] nixbot[3547]: {"ts": "2026-10-02T14:14:47.367Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1670gitlab # [ 329.482561] nixbot[3547]: {"ts": "2026-10-02T14:14:47.368Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1671gitlab # [ 329.485674] nixbot[3547]: {"ts": "2026-10-02T14:14:47.369Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1672gitlab # [ 329.488713] nixbot[3547]: {"ts": "2026-10-02T14:14:47.381Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1673gitlab # [ 329.491632] nixbot[3547]: {"ts": "2026-10-02T14:14:47.385Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1674gitlab # [ 329.494655] nixbot[3547]: {"ts": "2026-10-02T14:14:47.387Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1675gitlab # [ 329.497539] nixbot[3547]: {"ts": "2026-10-02T14:14:47.388Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1676gitlab # [ 329.500346] nixbot[3547]: {"ts": "2026-10-02T14:14:47.392Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1677gitlab # [ 329.503318] nixbot[3547]: {"ts": "2026-10-02T14:14:47.394Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1678gitlab # [ 329.506253] nixbot[3547]: {"ts": "2026-10-02T14:14:47.399Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1679gitlab # [ 329.509065] nixbot[3547]: {"ts": "2026-10-02T14:14:47.402Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1680gitlab # [ 329.881312] nixbot[3547]: INFO: Started server process [3547]1681gitlab # [ 329.882949] nixbot[3547]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1682gitlab # [ 329.903793] nixbot[3547]: INFO: ::ffff:127.0.0.1:46116 - "GET /health HTTP/1.1" 200 OK1683gitlab # [ 329.909641] systemd[1]: Started nixbot CI service.1684gitlab # [ 329.910973] systemd[1]: Reached target Multi-User System.1685gitlab # [ 329.912435] systemd[1]: Startup finished in 770ms (kernel) + 3.704s (initrd) + 5min 25.434s (userspace) = 5min 29.909s.1686gitlab: (finished: waiting for unit nixbot.service, in 1.06 seconds)1687gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1688gitlab # [ 330.002630] nixbot[3547]: INFO: ::ffff:127.0.0.1:46122 - "GET /health HTTP/1.1" 200 OK1689gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.04 seconds)1690subtest: project discovered and webhook registered1691gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1692gitlab # [ 330.277835] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 2341693gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.27 seconds)1694gitlab # [ 331.105243] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 12011695gitlab # [ 331.108482] nixbot[3547]: {"ts": "2026-10-02T14:14:49.014Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1696gitlab # [ 331.134330] nixbot[3547]: {"ts": "2026-10-02T14:14:49.041Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1697gitlab # [ 331.250323] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1061698gitlab # [ 331.254126] nixbot[3547]: {"ts": "2026-10-02T14:14:49.159Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1699gitlab # [ 331.258328] nixbot[3547]: {"ts": "2026-10-02T14:14:49.161Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1700gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1701gitlab # [ 331.423219] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 1091702gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.15 seconds)1703gitlab # [ 331.439336] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1811704gitlab # [ 331.441855] nixbot[3547]: {"ts": "2026-10-02T14:14:49.348Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1705gitlab # [ 331.617970] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1671706gitlab # [ 331.620740] nixbot[3547]: {"ts": "2026-10-02T14:14:49.526Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1707gitlab # [ 331.803480] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 1721708gitlab # [ 331.807487] nixbot[3547]: {"ts": "2026-10-02T14:14:49.712Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1709gitlab # [ 331.892584] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 831710gitlab # [ 331.896151] nixbot[3547]: {"ts": "2026-10-02T14:14:49.801Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1711gitlab # [ 332.000608] gitaly[924]: time="2026-10-02T14:14:49.907Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFFPHVGWARXZRR4YT4QFK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-02T14:15:19.690" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:14:49.891" grpc.time_ms=15.554 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 332.026176] gitaly[924]: time="2026-10-02T14:14:49.932Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFFPHVGWARXZRR4YT4QFK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:15:19.730" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:14:49.931" grpc.time_ms=0.944 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1713gitlab # [ 332.047149] gitaly[924]: time="2026-10-02T14:14:49.947Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFFPHVGWARXZRR4YT4QFK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-02T14:14:59.016" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:14:49.936" grpc.time_ms=10.034 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1714gitlab # [ 332.078851] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1761715gitlab # [ 332.082860] nixbot[3547]: {"ts": "2026-10-02T14:14:49.987Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1716gitlab # [ 332.195761] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1091717gitlab # [ 332.199191] nixbot[3547]: {"ts": "2026-10-02T14:14:50.104Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1718gitlab # [ 332.202179] nixbot[3547]: {"ts": "2026-10-02T14:14:50.109Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1719gitlab # [ 332.355769] gitaly[924]: time="2026-10-02T14:14:50.263Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFG272VGHP8A7BEEPV4H5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.251" grpc.time_ms=11.326 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1720gitlab # [ 332.373850] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 961721gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1722gitlab # [ 332.478111] gitaly[924]: time="2026-10-02T14:14:50.385Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFG5A5PRWNA3C2RFZ5Z72 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.375" pid=924 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1723gitlab # [ 332.490005] gitaly[924]: time="2026-10-02T14:14:50.385Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFG5A5PRWNA3C2RFZ5Z72 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.375" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1724gitlab # [ 332.505695] gitlab-workhorse[823]: time="2026-10-02T14:14:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131725gitlab # [ 332.507752] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 1041726gitlab # [ 332.510296] gitaly[924]: time="2026-10-02T14:14:50.386Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFG5A5PRWNA3C2RFZ5Z72 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.375" grpc.time_ms=10.087 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1727gitlab # [ 332.547644] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 841728gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds)1729(finished: subtest: project discovered and webhook registered, in 2.54 seconds)1730subtest: push delivers the webhook1731gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1732gitlab # [ 332.584119] gitaly[924]: time="2026-10-02T14:14:50.491Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFG8QJYXWY3ZBG2NEJWFH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.470" pid=924 remote_ip="2001:db8:1::1" request_sha=cfca229d095959b29f577241c8fd143eda3a1416 response_bytes=223 span.kind=server system=grpc1733gitlab # [ 332.601739] gitaly[924]: time="2026-10-02T14:14:50.491Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFG8QJYXWY3ZBG2NEJWFH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.470" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1734gitlab # [ 332.620387] gitlab-workhorse[823]: time="2026-10-02T14:14:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231735gitlab # [ 332.622620] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1001736gitlab # [ 332.624969] gitaly[924]: time="2026-10-02T14:14:50.491Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1138 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFG8QJYXWY3ZBG2NEJWFH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.470" grpc.time_ms=21.373 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1737gitlab # [ 332.692835] gitaly[924]: time="2026-10-02T14:14:50.599Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGCYM2212MYN6WR9EX6G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.589" grpc.time_ms=10.504 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1738gitlab # [ 332.715070] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 731739gitlab # [ 332.799001] gitaly[924]: time="2026-10-02T14:14:50.706Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGFA3R2NF22C0R4NRTK5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.695" pid=924 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1740gitlab # [ 332.816401] gitaly[924]: time="2026-10-02T14:14:50.706Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGFA3R2NF22C0R4NRTK5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.695" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1741gitlab # [ 332.838041] gitlab-workhorse[823]: time="2026-10-02T14:14:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671742gitlab # [ 332.840925] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1051743gitlab # [ 332.843651] gitaly[924]: time="2026-10-02T14:14:50.706Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGFA3R2NF22C0R4NRTK5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.695" grpc.time_ms=11.07 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1744gitlab # [ 332.907592] gitaly[924]: time="2026-10-02T14:14:50.814Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGKV6KJ0T2ZT1TFC6X5H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.805" grpc.time_ms=9.187 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1745gitlab # [ 332.931301] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 881746gitlab # [ 333.025369] gitaly[924]: time="2026-10-02T14:14:50.932Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGPPJN5TDH4VJCT0TJWW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.913" pid=924 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1747gitlab # [ 333.042543] gitaly[924]: time="2026-10-02T14:14:50.932Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGPPJN5TDH4VJCT0TJWW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.913" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1748gitlab # [ 333.066753] gitlab-workhorse[823]: time="2026-10-02T14:14:50Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091749gitlab # [ 333.070401] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:50 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 941750gitlab # [ 333.073380] gitaly[924]: time="2026-10-02T14:14:50.932Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGPPJN5TDH4VJCT0TJWW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:14:50.913" grpc.time_ms=19.143 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1751gitlab # [ 333.194401] gitaly[924]: time="2026-10-02T14:14:51.101Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:15:21.100" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:14:51.100" grpc.time_ms=0.849 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752gitlab # [ 333.240469] gitaly[924]: time="2026-10-02T14:14:51.147Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T14:15:21.135" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:14:51.136" grpc.time_ms=11.313 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 333.269168] gitaly[924]: time="2026-10-02T14:14:51.176Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:15:01.155" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:14:51.165" grpc.time_ms=10.902 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754gitlab # [ 333.292144] gitaly[924]: time="2026-10-02T14:14:51.198Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:15:01.178" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:14:51.188" grpc.time_ms=10.276 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755gitlab # [ 333.382792] gitaly[924]: time="2026-10-02T14:14:51.290Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:15:01.279" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:51.289" grpc.time_ms=0.517 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756gitlab # [ 333.402770] gitaly[924]: time="2026-10-02T14:14:51.310Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:15:01.289" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:14:51.299" grpc.time_ms=9.927 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1757gitlab # [ 333.444202] gitaly[924]: time="2026-10-02T14:14:51.351Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:14:51.350" grpc.time_ms=0.621 pid=924 span.kind=server system=grpc1758gitlab # [ 333.478654] gitaly[924]: time="2026-10-02T14:14:51.385Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:14:51.383" grpc.time_ms=2.433 pid=924 span.kind=server system=grpc1759gitlab # [ 333.495375] gitaly[924]: time="2026-10-02T14:14:51.402Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 diskcache=2ad24809-afdc-41af-8269-ce581d8dbbf5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:15:01.002" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:51.313" pid=924 ref=refs/keep-around/88b61d9ace3e1870aed365de3bf04ea0e6962e72 remote_ip="2001:db8:1::1" revision=88b61d9ace3e1870aed365de3bf04ea0e6962e72 span.kind=server system=grpc username=root1760gitlab # [ 333.513366] gitaly[924]: time="2026-10-02T14:14:51.403Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5434 command.oublock=8 command.real_time_ms=83 command.spawn_token_fork_ms=1 command.system_time_ms=33 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:15:01.002" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:14:51.313" grpc.time_ms=89.359 pid=924 ref=refs/keep-around/88b61d9ace3e1870aed365de3bf04ea0e6962e72 remote_ip="2001:db8:1::1" revision=88b61d9ace3e1870aed365de3bf04ea0e6962e72 span.kind=server system=grpc username=root1761gitlab # [ 333.532666] gitaly[924]: time="2026-10-02T14:14:51.409Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:15:21.207" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:14:51.408" grpc.time_ms=0.589 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 333.709183] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:14:51 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 6051763gitlab # [ 333.714161] nixbot[3547]: {"ts": "2026-10-02T14:14:51.617Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 \"HTTP/1.1 201 Created\""}1764gitlab # [ 333.722772] nixbot[3547]: {"ts": "2026-10-02T14:14:51.629Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-88b61d9ace3e-0"}1765gitlab # [ 333.921978] nixbot[3547]: {"ts": "2026-10-02T14:14:51.829Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-88b61d9ace3e-0", "argv0": "bwrap"}1766gitlab # [ 334.060664] systemd[1]: Started Nix Daemon.1767gitlab # [ 334.143287] nix-daemon[3710]: accepted connection from pid 3705, user nixbot1768gitlab # [ 338.803556] nix-daemon[3710]: accepted connection from pid 3753, user nixbot1769gitlab # [ 338.824572] nix-daemon[3710]: accepted connection from pid 3705, user nixbot1770gitlab # [ 343.303428] nix-daemon[3710]: accepted connection from pid 3705, user nixbot1771gitlab # [ 343.463767] gitaly[924]: time="2026-10-02T14:15:01.370Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFTWMW1EXDCB3CQ9DDSK6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:15:31.064" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:15:01.364" grpc.time_ms=5.759 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772gitlab # [ 343.640256] gitaly[924]: time="2026-10-02T14:15:01.547Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFTWMW1EXDCB3CQ9DDSK6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:15:11.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:15:01.534" grpc.time_ms=12.317 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 343.795951] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 4331774gitlab # [ 343.801144] nixbot[3547]: {"ts": "2026-10-02T14:15:01.704Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 \"HTTP/1.1 201 Created\""}1775gitlab # [ 343.877641] gitaly[924]: time="2026-10-02T14:15:01.784Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFVAESQVC8Y778REAVK37 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:15:31.483" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:15:01.783" grpc.time_ms=0.941 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1776gitlab # [ 343.926978] gitaly[924]: time="2026-10-02T14:15:01.834Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFVAESQVC8Y778REAVK37 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:15:11.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:15:01.824" grpc.time_ms=9.934 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1777gitlab # [ 343.992539] nixbot[3547]: {"ts": "2026-10-02T14:15:01.899Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 \"HTTP/1.1 201 Created\""}1778gitlab # [ 344.000500] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:01 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1911779gitlab # [ 344.087792] gitaly[924]: time="2026-10-02T14:15:01.995Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFVH02QNB9HVZAA7VTF66 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:15:31.693" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:15:01.994" grpc.time_ms=0.849 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 344.141112] gitaly[924]: time="2026-10-02T14:15:02.048Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFVH02QNB9HVZAA7VTF66 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:15:12.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:15:02.038" grpc.time_ms=9.7 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1781gitlab # [ 344.215852] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2021782gitlab # [ 344.220040] nixbot[3547]: {"ts": "2026-10-02T14:15:02.124Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 \"HTTP/1.1 201 Created\""}1783gitlab # [ 344.688933] gitaly[924]: time="2026-10-02T14:15:02.595Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFW3B5JFQ1KVFAT3KNAV3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:02.582" grpc.time_ms=12.28 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1784gitlab # [ 344.713848] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1121785gitlab # [ 344.815420] gitaly[924]: time="2026-10-02T14:15:02.722Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFW6YQT7F9TM944B25ZW6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:02.708" pid=924 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1786gitlab # [ 344.835664] gitaly[924]: time="2026-10-02T14:15:02.722Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFW6YQT7F9TM944B25ZW6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:02.708" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1787gitlab # [ 344.859745] gitlab-workhorse[823]: time="2026-10-02T14:15:02Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671788gitlab # [ 344.862989] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:02 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1011789gitlab # [ 344.866117] gitaly[924]: time="2026-10-02T14:15:02.723Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFW6YQT7F9TM944B25ZW6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:02.708" grpc.time_ms=14.18 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1790gitlab # [ 344.960622] gitaly[924]: time="2026-10-02T14:15:02.867Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFWBFRXZS9KMEDKJK4SCQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:02.853" grpc.time_ms=14.555 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1791gitlab # [ 344.983283] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:02 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1001792gitlab # [ 345.239896] gitaly[924]: time="2026-10-02T14:15:03.146Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFWEP7CX93PH1G30AEAYB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:03.133" pid=924 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1793gitlab # [ 345.261915] gitaly[924]: time="2026-10-02T14:15:03.147Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFWEP7CX93PH1G30AEAYB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:03.133" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1794gitlab # [ 345.284165] gitlab-workhorse[823]: time="2026-10-02T14:15:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671795gitlab # [ 345.287057] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:15:03 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 2771796gitlab # [ 345.290349] gitaly[924]: time="2026-10-02T14:15:03.147Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFWEP7CX93PH1G30AEAYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:15:03.133" grpc.time_ms=13.606 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1797gitlab # [ 364.340420] sidekiq-cluster[3422]: 2026-10-02T14:15:22.246Z pid=3422 tid=yra INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3422, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>424075264}1798gitlab # [ 364.618250] sidekiq-cluster[3422]: W, [2026-10-02T14:15:22.525429 #3422] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1799gitlab # [ 364.766858] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:15:22.674Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1800gitlab # [ 368.121860] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:15:26.028Z","message":"GitLab reliable fetch activated!"}1801gitlab # [ 368.122147] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:15:26.028Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1802gitlab # [ 404.699525] postgres[4313]: [4313] ERROR: relation "table_without_models" does not exist1803gitlab # [ 404.699946] postgres[4313]: [4313] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1804gitlab # [ 409.604185] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:07.507Z","message":"Booted Rails 7.2.3.1 application in production environment"}1805gitlab # [ 409.604594] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:07.511Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1806gitlab # [ 409.604675] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:07.512Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1807gitlab # [ 409.604769] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:07.512Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1808gitlab # [ 409.617526] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:07.515Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1809gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 77.85 seconds)1810gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1811gitlab # [ 410.553662] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:08 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 941812gitlab # [ 410.718437] gitaly[924]: time="2026-10-02T14:16:08.625Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWGDAD3JXCE1J3HR3BS5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:08.612" grpc.time_ms=12.913 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1813gitlab # Enumerating objects: 5, done.1814gitlab # [ 410.737004] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:08 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 1631815gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1816gitlab # Delta compression using up to 4 threads1817gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1818gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 272 bytes | 136.00 KiB/s, done.1819gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1820gitlab # [ 411.109496] gitaly[924]: time="2026-10-02T14:16:09.016Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T14:16:38.798" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T14:16:08.998" grpc.time_ms=17.629 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1821gitlab # [ 411.171191] gitaly[924]: time="2026-10-02T14:16:09.078Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ duration_ms=226ns method=POST pid=924 status=200 url="http://unix/api/v4/internal/allowed"1822gitlab # [ 411.243784] gitaly[924]: time="2026-10-02T14:16:09.151Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ duration_ms=71ns method=POST pid=924 status=200 url="http://unix/api/v4/internal/pre_receive"1823gitlab # [ 411.252252] gitaly[924]: time="2026-10-02T14:16:09.151Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:08.850" grpc.time_ms=301.518 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1824gitlab # [ 411.322823] gitaly[924]: time="2026-10-02T14:16:09.230Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:09.229" grpc.time_ms=0.869 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1825gitlab # [ 411.381580] gitaly[924]: time="2026-10-02T14:16:09.288Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:09.287" grpc.time_ms=1.274 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1826gitlab # [ 411.434528] gitaly[924]: time="2026-10-02T14:16:09.341Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:09.340" grpc.time_ms=1.293 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1827gitlab # [ 411.574897] gitaly[924]: time="2026-10-02T14:16:09.482Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ duration_ms=86ns method=POST pid=924 status=200 url="http://unix/api/v4/internal/post_receive"1828gitlab # [ 411.583328] gitaly[924]: time="2026-10-02T14:16:09.483Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:09.394" grpc.time_ms=87.945 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1829gitlab # To http://gitlab/root/test-flake.git1830gitlab # [ 411.601607] gitaly[924]: time="2026-10-02T14:16:09.495Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ diskcache=99b282c2-8974-4854-8ea2-5d1908cd69ef grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:08.764" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1831gitlab # 88b61d9..45e21e5 master -> master1832gitlab # [ 411.620157] gitlab-workhorse[823]: time="2026-10-02T14:16:09Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521833gitlab # [ 411.623428] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:09 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 8331834gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.23 seconds)1835gitlab: must succeed: git -C /tmp/test-flake rev-parse master1836gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds)1837gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1838gitlab # [ 411.626875] gitaly[924]: time="2026-10-02T14:16:09.495Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=285 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13701 command.oublock=32 command.real_time_ms=724 command.spawn_token_fork_ms=0 command.system_time_ms=142 command.user_time_ms=143 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=604 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:08.764" grpc.time_ms=730.943 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1839gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.10 seconds)1840gitlab # [ 411.739788] nixbot[3547]: INFO: ::ffff:127.0.0.1:54470 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1841gitlab # [ 412.555559] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:10.461Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"06be51c42b4a7720f4511319","created_at":"2026-10-02T14:14:35.953Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:35.988Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-06be51c42b4a7720f4511319: start","job_status":"start","queue_duration_s":94.473163,"scheduling_latency_s":94.473235}1842gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1843gitlab # [ 412.779545] nixbot[3547]: INFO: ::ffff:127.0.0.1:54474 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1844gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.04 seconds)1845gitlab # [ 412.816472] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:10.722Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"06be51c42b4a7720f4511319","created_at":"2026-10-02T14:14:35.953Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:35.988Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-06be51c42b4a7720f4511319: done: 0.261438 sec","job_status":"done","queue_duration_s":94.473163,"scheduling_latency_s":94.473235,"redis_calls":23,"redis_duration_s":0.006563,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.003259,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001748,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.144465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14424936,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.261438,"completed_at":"2026-10-02T14:16:10.722Z","load_balancing_strategy":"primary","db_duration_s":0.074782,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1846gitlab # [ 412.886065] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:10.725Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3baf2021c3d0588fb08dd008","created_at":"2026-10-02T14:14:36.002Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.008Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3baf2021c3d0588fb08dd008: start","job_status":"start","queue_duration_s":94.716834,"scheduling_latency_s":94.716871}1847gitlab # [ 412.949797] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:10.856Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"3baf2021c3d0588fb08dd008","created_at":"2026-10-02T14:14:36.002Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.008Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-3baf2021c3d0588fb08dd008: done: 0.131419 sec","job_status":"done","queue_duration_s":94.716834,"scheduling_latency_s":94.716871,"redis_calls":14,"redis_duration_s":0.003652,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001009,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0015,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.096547,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09979942,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.131419,"completed_at":"2026-10-02T14:16:10.856Z","load_balancing_strategy":"primary","db_duration_s":0.026678,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1848gitlab # [ 413.018092] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:10.858Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0b14589fab5e3cfed0cc4bb5","created_at":"2026-10-02T14:14:36.634Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.642Z","job_size_bytes":5,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0b14589fab5e3cfed0cc4bb5: start","job_status":"start","queue_duration_s":94.216202,"scheduling_latency_s":94.216347}1849gitlab # [ 413.146832] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:11.053Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"0b14589fab5e3cfed0cc4bb5","created_at":"2026-10-02T14:14:36.634Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.642Z","job_size_bytes":5,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-0b14589fab5e3cfed0cc4bb5: done: 0.195252 sec","job_status":"done","queue_duration_s":94.216202,"scheduling_latency_s":94.216347,"redis_calls":12,"redis_duration_s":0.002582,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000703,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001491,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.158257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028092,"duration_s":0.195252,"completed_at":"2026-10-02T14:16:11.053Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031516,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1850gitlab # [ 413.218062] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:11.055Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f8273034fa15e24b2a5d5404","created_at":"2026-10-02T14:14:36.675Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.680Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f8273034fa15e24b2a5d5404: start","job_status":"start","queue_duration_s":94.374542,"scheduling_latency_s":94.374566}1851gitlab # [ 413.379896] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:11.286Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f8273034fa15e24b2a5d5404","created_at":"2026-10-02T14:14:36.675Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.680Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f8273034fa15e24b2a5d5404: done: 0.23116 sec","job_status":"done","queue_duration_s":94.374542,"scheduling_latency_s":94.374566,"redis_calls":11,"redis_duration_s":0.002403,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000577,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001394,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000432,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.192,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.192,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.156984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040322,"duration_s":0.23116,"completed_at":"2026-10-02T14:16:11.286Z","load_balancing_strategy":"primary","db_duration_s":0.078231,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1852gitlab # [ 413.447568] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:11.288Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"3ed336d0ed19290c838fd2fe","created_at":"2026-10-02T14:14:36.684Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.689Z","job_size_bytes":89,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-3ed336d0ed19290c838fd2fe: start","job_status":"start","queue_duration_s":94.59931,"scheduling_latency_s":94.599332}1853gitlab # [ 413.467871] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:11.330Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"3ed336d0ed19290c838fd2fe","created_at":"2026-10-02T14:14:36.684Z","trace_propagation_headers":{"sentry-trace":"b404313cbde14d4ebe4a9614eb285a9e-3592debb069d4a10","baggage":"sentry-trace_id=b404313cbde14d4ebe4a9614eb285a9e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFF13A00Q2V3WJRV973VVR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:36.689Z","job_size_bytes":89,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-3ed336d0ed19290c838fd2fe: done: 0.042242 sec","job_status":"done","queue_duration_s":94.59931,"scheduling_latency_s":94.599332,"redis_calls":11,"redis_duration_s":0.002349,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036078,"duration_s":0.042242,"completed_at":"2026-10-02T14:16:11.330Z","load_balancing_strategy":"primary","db_duration_s":0.007051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1854gitlab # [ 413.516435] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:11.332Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 88b61d9ace3e1870aed365de3bf04ea0e6962e72 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"0e3b00938ff418117559a087","created_at":"2026-10-02T14:14:45.876Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-206dceaf163542da","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:45.896Z","job_size_bytes":185,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-0e3b00938ff418117559a087: start","job_status":"start","queue_duration_s":85.436634,"scheduling_latency_s":85.436657}1855gitlab # [ 413.533947] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:11.353Z","message":"changes: \"0000000000000000000000000000000000000000 88b61d9ace3e1870aed365de3bf04ea0e6962e72 refs/heads/master\\n\""}1856gitlab # [ 413.537319] gitaly[924]: time="2026-10-02T14:16:11.399Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:21.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:11.387" grpc.time_ms=11.772 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857gitlab # [ 413.552544] gitaly[924]: time="2026-10-02T14:16:11.416Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=5 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:21.025" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:11.405" grpc.time_ms=10.744 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab # [ 413.582908] gitaly[924]: time="2026-10-02T14:16:11.490Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T14:16:21.095" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:11.476" grpc.time_ms=13.905 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1859gitlab # [ 413.634535] gitaly[924]: time="2026-10-02T14:16:11.541Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:11.541" grpc.time_ms=0.556 pid=924 span.kind=server system=grpc1860gitlab # [ 413.667967] gitaly[924]: time="2026-10-02T14:16:11.575Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:11.574" grpc.time_ms=0.553 pid=924 span.kind=server system=grpc1861gitlab # [ 413.682728] gitaly[924]: time="2026-10-02T14:16:11.583Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S diskcache=9c574c2d-b742-428c-ab41-d35000f713ce grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:16:21.127" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:11.507" pid=924 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1862gitlab # [ 413.696675] gitaly[924]: time="2026-10-02T14:16:11.583Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=82 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5155 command.oublock=8 command.real_time_ms=70 command.spawn_token_fork_ms=0 command.system_time_ms=39 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:16:21.127" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:11.507" grpc.time_ms=75.476 pid=924 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1863gitlab # [ 413.713397] gitaly[924]: time="2026-10-02T14:16:11.604Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T14:16:21.223" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:11.603" grpc.time_ms=0.192 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864gitlab # [ 413.726488] gitaly[924]: time="2026-10-02T14:16:11.624Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T14:16:21.229" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:11.609" grpc.time_ms=14.318 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1866gitlab # [ 413.822150] nixbot[3547]: INFO: ::ffff:127.0.0.1:54484 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1867gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.04 seconds)1868gitlab # [ 413.851243] gitaly[924]: time="2026-10-02T14:16:11.758Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-02T14:16:41.050" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:11.750" grpc.time_ms=7.572 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869gitlab # [ 413.896999] gitaly[924]: time="2026-10-02T14:16:11.804Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:16:21.003" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:11.803" grpc.time_ms=0.385 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870gitlab # [ 413.914507] gitaly[924]: time="2026-10-02T14:16:11.814Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T14:16:21.007" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T14:16:11.807" grpc.time_ms=7.14 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1871gitlab # [ 413.937681] gitaly[924]: time="2026-10-02T14:16:11.843Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:21.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:11.832" grpc.time_ms=10.573 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1872gitlab # [ 413.958184] gitaly[924]: time="2026-10-02T14:16:11.860Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:21.049" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:11.849" grpc.time_ms=11.032 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 413.987743] gitaly[924]: time="2026-10-02T14:16:11.895Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T14:16:41.177" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:11.877" grpc.time_ms=17.288 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 414.569845] gitaly[924]: time="2026-10-02T14:16:12.476Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T14:16:22.093" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:12.474" grpc.time_ms=2.064 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875gitlab # [ 414.619295] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:12.525Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 88b61d9ace3e1870aed365de3bf04ea0e6962e72 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"0e3b00938ff418117559a087","created_at":"2026-10-02T14:14:45.876Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-206dceaf163542da","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:14:45.896Z","job_size_bytes":185,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-0e3b00938ff418117559a087: done: 1.192621 sec","job_status":"done","queue_duration_s":85.436634,"scheduling_latency_s":85.436657,"gitaly_calls":13,"gitaly_duration_s":0.22891,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.048161999999999996,"redis_read_bytes":2103,"redis_write_bytes":21751,"redis_cache_calls":15,"redis_cache_duration_s":0.004811,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.007272,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001918,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7839,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004007,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.017483,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.012671,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.287,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.327,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.068,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.862212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00086286,"exclusive_lock_hold_duration_s":0.0339147,"duration_s":1.192621,"completed_at":"2026-10-02T14:16:12.525Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.117484,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1876gitlab # [ 414.698119] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:12.527Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"50effea56102f96f4e44dc04","created_at":"2026-10-02T14:14:51.414Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-1d8d990c1a0a4bb4","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:51.425Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-50effea56102f96f4e44dc04: start","job_status":"start","queue_duration_s":81.102327,"scheduling_latency_s":81.102359}1877gitlab # [ 414.753194] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:12.659Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"50effea56102f96f4e44dc04","created_at":"2026-10-02T14:14:51.414Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-1d8d990c1a0a4bb4","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:51.425Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-50effea56102f96f4e44dc04: done: 0.132338 sec","job_status":"done","queue_duration_s":81.102327,"scheduling_latency_s":81.102359,"redis_calls":16,"redis_duration_s":0.005789,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002764,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002287,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000738,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.095382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069572,"duration_s":0.132338,"completed_at":"2026-10-02T14:16:12.659Z","load_balancing_strategy":"primary","db_duration_s":0.028165,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1878gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1879gitlab # [ 414.819681] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:12.662Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f15c4733e73c879bea2199b4","created_at":"2026-10-02T14:14:51.426Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-1d8d990c1a0a4bb4","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:51.436Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f15c4733e73c879bea2199b4: start","job_status":"start","queue_duration_s":81.226472,"scheduling_latency_s":81.226506}1880gitlab # [ 414.863929] nixbot[3547]: INFO: ::ffff:127.0.0.1:54496 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1881gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.04 seconds)1882gitlab # [ 414.889875] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:12.796Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f15c4733e73c879bea2199b4","created_at":"2026-10-02T14:14:51.426Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-1d8d990c1a0a4bb4","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:51.436Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f15c4733e73c879bea2199b4: done: 0.13423 sec","job_status":"done","queue_duration_s":81.226472,"scheduling_latency_s":81.226506,"redis_calls":21,"redis_duration_s":0.005091,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.00211,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001674,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001307,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039546,"duration_s":0.13423,"completed_at":"2026-10-02T14:16:12.796Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1883gitlab # [ 414.961392] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:12.798Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"248508239a839118025f9eb4","created_at":"2026-10-02T14:14:51.589Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-1d8d990c1a0a4bb4","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:51.596Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-248508239a839118025f9eb4: start","job_status":"start","queue_duration_s":81.202,"scheduling_latency_s":81.202026}1884gitlab # [ 415.303273] gitaly[924]: time="2026-10-02T14:16:13.210Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:13.209" grpc.time_ms=0.602 pid=924 span.kind=server system=grpc1885gitlab # [ 415.336658] gitaly[924]: time="2026-10-02T14:16:13.243Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:13.243" grpc.time_ms=0.546 pid=924 span.kind=server system=grpc1886gitlab # [ 415.372789] gitaly[924]: time="2026-10-02T14:16:13.280Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:13.278" grpc.time_ms=1.939 pid=924 span.kind=server system=grpc1887gitlab # [ 415.390164] gitaly[924]: time="2026-10-02T14:16:13.292Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 diskcache=60099af2-7f91-4850-a292-15718437cee1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:16:43.074" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:13.174" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab # [ 415.406147] gitaly[924]: time="2026-10-02T14:16:13.293Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=128 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7576 command.oublock=0 command.real_time_ms=112 command.spawn_token_fork_ms=0 command.system_time_ms=87 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:16:43.074" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:13.174" grpc.time_ms=118.522 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab # [ 415.598529] gitaly[924]: time="2026-10-02T14:16:13.505Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:16:43.403" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:13.503" grpc.time_ms=1.404 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 415.670010] gitaly[924]: time="2026-10-02T14:16:13.577Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFGW27D61DW6YVDZ7WFB8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:43.475" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:13.575" grpc.time_ms=1.111 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891gitlab # [ 415.706844] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:13.612Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"248508239a839118025f9eb4","created_at":"2026-10-02T14:14:51.589Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-1d8d990c1a0a4bb4","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:14:51.596Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-248508239a839118025f9eb4: done: 0.814097 sec","job_status":"done","queue_duration_s":81.202,"scheduling_latency_s":81.202026,"gitaly_calls":3,"gitaly_duration_s":0.132982,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.036090000000000004,"redis_read_bytes":455,"redis_write_bytes":27994,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00176,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.003008,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.012819,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005263,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14488,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.007157,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001058,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000633,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002632,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.077,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.03,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.034,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.037,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.585323,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.73259608,"duration_s":0.814097,"completed_at":"2026-10-02T14:16:13.612Z","load_balancing_strategy":"primary","db_duration_s":0.09249,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1892gitlab # [ 415.776334] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:13.615Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","88b61d9ace3e1870aed365de3bf04ea0e6962e72 45e21e5de49e68885eab25da3122e70bb390a5df refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b19f582706948a12201fa0a3","created_at":"2026-10-02T14:16:09.448Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-fd62321701fe4791","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:09.454Z","job_size_bytes":185,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b19f582706948a12201fa0a3: start","job_status":"start","queue_duration_s":4.160784,"scheduling_latency_s":4.160821}1893gitlab # [ 415.795724] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:13.639Z","message":"changes: \"88b61d9ace3e1870aed365de3bf04ea0e6962e72 45e21e5de49e68885eab25da3122e70bb390a5df refs/heads/master\\n\""}1894gitlab # [ 415.798441] gitaly[924]: time="2026-10-02T14:16:13.684Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:23.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:13.672" grpc.time_ms=12.678 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1895gitlab # [ 415.812595] gitaly[924]: time="2026-10-02T14:16:13.705Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T14:16:23.022" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:13.692" grpc.time_ms=12.222 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1896gitlab # [ 415.826745] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:13.712Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"696fa734de7007a1490575ed","created_at":"2026-10-02T14:16:13.709Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-c0a8dd867b5f4ea0","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"49c0dde0a80b0c5272df038a","job_size_bytes":46,"pid":3422,"job_status":"deduplicated","message":"ProjectCacheWorker JID-696fa734de7007a1490575ed: deduplicated: until executing","deduplication.type":"until executing"}1897gitlab # [ 415.842567] gitaly[924]: time="2026-10-02T14:16:13.749Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T14:16:23.065" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:13.736" grpc.time_ms=12.922 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1899gitlab # [ 415.863126] gitaly[924]: time="2026-10-02T14:16:13.770Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-02T14:16:43.056" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:13.756" grpc.time_ms=13.492 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.04 seconds)1901gitlab # [ 415.895280] gitaly[924]: time="2026-10-02T14:16:13.802Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T14:16:23.124" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T14:16:13.794" grpc.time_ms=7.992 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902gitlab # [ 415.913856] nixbot[3547]: INFO: ::ffff:127.0.0.1:54500 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1903gitlab # [ 415.923763] gitaly[924]: time="2026-10-02T14:16:13.831Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:23.019" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:13.819" grpc.time_ms=11.118 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 416.088286] gitaly[924]: time="2026-10-02T14:16:13.995Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T14:16:23.193" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:13.993" grpc.time_ms=2.049 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 416.132875] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.039Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","88b61d9ace3e1870aed365de3bf04ea0e6962e72 45e21e5de49e68885eab25da3122e70bb390a5df refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"b19f582706948a12201fa0a3","created_at":"2026-10-02T14:16:09.448Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-fd62321701fe4791","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:09.454Z","job_size_bytes":185,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-b19f582706948a12201fa0a3: done: 0.423766 sec","job_status":"done","queue_duration_s":4.160784,"scheduling_latency_s":4.160821,"gitaly_calls":7,"gitaly_duration_s":0.101468,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.030979,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.003046,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.00136,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005815,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.008714,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.012044,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.04,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.04,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.287992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0017686,"exclusive_lock_hold_duration_s":0.02827274,"duration_s":0.423766,"completed_at":"2026-10-02T14:16:14.039Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.039895,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1906gitlab # [ 416.198375] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.041Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"49c0dde0a80b0c5272df038a","created_at":"2026-10-02T14:16:11.419Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:11.440Z","job_size_bytes":46,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-49c0dde0a80b0c5272df038a: start","job_status":"start","queue_duration_s":2.600956,"scheduling_latency_s":2.601002}1907gitlab # [ 416.224895] gitaly[924]: time="2026-10-02T14:16:14.132Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-02T20:16:14.125" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:14.125" grpc.time_ms=5.861 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 416.249134] gitaly[924]: time="2026-10-02T14:16:14.156Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T14:16:44.044" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:14.144" grpc.time_ms=11.491 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 416.295965] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.202Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"49c0dde0a80b0c5272df038a","created_at":"2026-10-02T14:16:11.419Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:11.440Z","job_size_bytes":46,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-49c0dde0a80b0c5272df038a: done: 0.160771 sec","job_status":"done","queue_duration_s":2.600956,"scheduling_latency_s":2.601002,"gitaly_calls":2,"gitaly_duration_s":0.024471,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.008518,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001646,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000673,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001938,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003336,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.112479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038128,"duration_s":0.160771,"completed_at":"2026-10-02T14:16:14.202Z","load_balancing_strategy":"primary","db_duration_s":0.015481,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1910gitlab # [ 416.365855] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.205Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b91a82e56d323ca1acc908a7","created_at":"2026-10-02T14:16:11.461Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:11.470Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b91a82e56d323ca1acc908a7: start","job_status":"start","queue_duration_s":2.735183,"scheduling_latency_s":2.73523}1911gitlab # [ 416.386275] gitaly[924]: time="2026-10-02T14:16:14.281Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S diskcache=7321cc09-0fb2-45cc-a495-497294f12604 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:16:14.252" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:14.253" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 416.398922] gitaly[924]: time="2026-10-02T14:16:14.282Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1173 command.oublock=0 command.real_time_ms=33 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:16:14.252" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:14.253" grpc.time_ms=28.362 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 416.440826] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.347Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b91a82e56d323ca1acc908a7","created_at":"2026-10-02T14:16:11.461Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:11.470Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b91a82e56d323ca1acc908a7: done: 0.142136 sec","job_status":"done","queue_duration_s":2.735183,"scheduling_latency_s":2.73523,"gitaly_calls":1,"gitaly_duration_s":0.032595,"redis_calls":15,"redis_duration_s":0.004885,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00073,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00174,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000772,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001643,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.017,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.033,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09730804,"duration_s":0.142136,"completed_at":"2026-10-02T14:16:14.347Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1914gitlab # [ 416.497833] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.349Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a40a2bb594f3c256714a905c","created_at":"2026-10-02T14:16:11.596Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:11.599Z","job_size_bytes":23,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a40a2bb594f3c256714a905c: start","job_status":"start","queue_duration_s":2.750144,"scheduling_latency_s":2.750175}1915gitlab # [ 416.514409] gitaly[924]: time="2026-10-02T14:16:14.400Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T14:16:44.287" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:14.387" grpc.time_ms=12.072 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 416.528358] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.415Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a40a2bb594f3c256714a905c","created_at":"2026-10-02T14:16:11.596Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:11.599Z","job_size_bytes":23,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a40a2bb594f3c256714a905c: done: 0.065586 sec","job_status":"done","queue_duration_s":2.750144,"scheduling_latency_s":2.750175,"gitaly_calls":1,"gitaly_duration_s":0.01514,"redis_calls":16,"redis_duration_s":0.004753,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000397,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001864,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001698,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000794,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065586,"completed_at":"2026-10-02T14:16:14.415Z","load_balancing_strategy":"primary","db_duration_s":0.003648,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1917gitlab # [ 416.575582] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:14.417Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7ea94896ca4108e20911b648","created_at":"2026-10-02T14:16:12.262Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b670b060225238f4fc89ab86014de04bccea14f25e5ee6dcb5ec1cc2f8c500","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:12.271Z","job_size_bytes":467,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7ea94896ca4108e20911b648: start","job_status":"start","queue_duration_s":2.14594,"scheduling_latency_s":2.145975}1918gitlab # [ 416.593928] gitaly[924]: time="2026-10-02T14:16:14.454Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:44.353" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:14.453" grpc.time_ms=0.895 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 416.608695] gitaly[924]: time="2026-10-02T14:16:14.469Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:16:24.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:14.458" grpc.time_ms=11.433 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 416.740902] gitaly[924]: time="2026-10-02T14:16:14.648Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:16:44.546" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:14.646" grpc.time_ms=1.189 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921gitlab # [ 416.762794] gitaly[924]: time="2026-10-02T14:16:14.657Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=47 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T14:16:44.553" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:14.653" grpc.time_ms=3.459 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922gitlab # [ 416.872979] gitaly[924]: time="2026-10-02T14:16:14.780Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:24.097" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:14.767" grpc.time_ms=12.761 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1924gitlab # [ 416.955510] nixbot[3547]: INFO: ::ffff:127.0.0.1:54504 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1925gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.05 seconds)1926gitlab # [ 417.805417] gitaly[924]: time="2026-10-02T14:16:15.712Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T14:16:45.592" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:15.692" grpc.time_ms=19.456 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1927gitlab # [ 417.827929] gitaly[924]: time="2026-10-02T14:16:15.735Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFFAHW9TRM88X23S3NB33S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T14:16:45.616" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:15.716" grpc.time_ms=18.733 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab # [ 417.885040] sidekiq-cluster[3422]: {"severity":"WARN","time":"2026-10-02T14:16:15.791Z","class":"CreatePipelineWorker","correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"88b61d9ace3e1870aed365de3bf04ea0e6962e72","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"88b61d9ace3e1870aed365de3bf04ea0e6962e72","ref":"refs/heads/master"},"retry":0}1929gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1930gitlab # [ 417.919013] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:15.802Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"7ea94896ca4108e20911b648","created_at":"2026-10-02T14:16:12.262Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b670b060225238f4fc89ab86014de04bccea14f25e5ee6dcb5ec1cc2f8c500","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:12.271Z","job_size_bytes":467,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-7ea94896ca4108e20911b648: done: 1.384474 sec","job_status":"done","queue_duration_s":2.14594,"scheduling_latency_s":2.145975,"gitaly_calls":7,"gitaly_duration_s":0.095331,"redis_calls":51,"redis_duration_s":0.013462,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.001268,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005999,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00352,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001302,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.225497,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041424,"duration_s":1.384474,"completed_at":"2026-10-02T14:16:15.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.071407,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1931gitlab # [ 417.991487] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:15.804Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"db9d37f3-b115-4b38-a5f9-cc1303e61b37\"}"],"class":"WebHookWorker","jid":"84aaafd9860e1956274efe5d","created_at":"2026-10-02T14:16:12.437Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8386280a72fe0b0ff0a9d5eb2a787b05269d764df2a225bbe8b2aa4b3125cf5d","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:12.443Z","job_size_bytes":1751,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-84aaafd9860e1956274efe5d: start","job_status":"start","queue_duration_s":3.360469,"scheduling_latency_s":3.360496}1932gitlab # [ 418.073873] nixbot[3547]: INFO: ::ffff:127.0.0.1:54520 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1933gitlab # [ 418.077686] nixbot[3547]: INFO: ::ffff:127.0.0.1:54516 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1934gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.12 seconds)1935gitlab # [ 418.099929] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.006Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"db9d37f3-b115-4b38-a5f9-cc1303e61b37\"}"],"class":"WebHookWorker","jid":"84aaafd9860e1956274efe5d","created_at":"2026-10-02T14:16:12.437Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-a08c95888411412a","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8386280a72fe0b0ff0a9d5eb2a787b05269d764df2a225bbe8b2aa4b3125cf5d","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:12.443Z","job_size_bytes":1751,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-84aaafd9860e1956274efe5d: done: 0.202143 sec","job_status":"done","queue_duration_s":3.360469,"scheduling_latency_s":3.360496,"redis_calls":19,"redis_duration_s":0.004721,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001133,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000496,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0022,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07634798000003684,"cpu_s":0.109062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037416,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.202143,"completed_at":"2026-10-02T14:16:16.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004282,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1936gitlab # [ 418.174070] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.008Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3af854edb9227806e2e69754","created_at":"2026-10-02T14:16:13.145Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.154Z","job_size_bytes":23,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3af854edb9227806e2e69754: start","job_status":"start","queue_duration_s":2.853889,"scheduling_latency_s":2.85392}1937gitlab # [ 418.203724] gitaly[924]: time="2026-10-02T14:16:16.111Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ3WDXCG2BR5EJWXN6B00 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:16.100" grpc.time_ms=10.544 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1938gitlab # [ 418.221227] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1061939gitlab # [ 418.241369] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.148Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3af854edb9227806e2e69754","created_at":"2026-10-02T14:16:13.145Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.154Z","job_size_bytes":23,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3af854edb9227806e2e69754: done: 0.139253 sec","job_status":"done","queue_duration_s":2.853889,"scheduling_latency_s":2.85392,"redis_calls":10,"redis_duration_s":0.003009,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000503,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001811,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058822,"duration_s":0.139253,"completed_at":"2026-10-02T14:16:16.147Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023185,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1940gitlab # [ 418.298717] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.151Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e633c3250575319bf869de39","created_at":"2026-10-02T14:16:13.302Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.312Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e633c3250575319bf869de39: start","job_status":"start","queue_duration_s":2.838559,"scheduling_latency_s":2.838591}1941gitlab # [ 418.318127] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.216Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e633c3250575319bf869de39","created_at":"2026-10-02T14:16:13.302Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.312Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e633c3250575319bf869de39: done: 0.065207 sec","job_status":"done","queue_duration_s":2.838559,"scheduling_latency_s":2.838591,"redis_calls":10,"redis_duration_s":0.003528,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000561,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002104,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052242,"duration_s":0.065207,"completed_at":"2026-10-02T14:16:16.216Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1942gitlab # [ 418.366343] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.219Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4fafb36a4c7debddb5fa35f3","created_at":"2026-10-02T14:16:13.337Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.349Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4fafb36a4c7debddb5fa35f3: start","job_status":"start","queue_duration_s":2.869914,"scheduling_latency_s":2.869948}1943gitlab # [ 418.384195] gitaly[924]: time="2026-10-02T14:16:16.236Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ3ZTWXNTHAG41MJQY9JD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:16.222" pid=924 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1944gitlab # [ 418.395631] gitlab-workhorse[823]: time="2026-10-02T14:16:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671945gitlab # [ 418.397819] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1141946gitlab # [ 418.399944] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.293Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"4fafb36a4c7debddb5fa35f3","created_at":"2026-10-02T14:16:13.337Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.349Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-4fafb36a4c7debddb5fa35f3: done: 0.074848 sec","job_status":"done","queue_duration_s":2.869914,"scheduling_latency_s":2.869948,"redis_calls":10,"redis_duration_s":0.003345,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000741,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001943,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000661,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041012,"duration_s":0.074848,"completed_at":"2026-10-02T14:16:16.293Z","load_balancing_strategy":"primary","db_duration_s":0.00996,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1947gitlab # [ 418.449476] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.296Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"68a5d940dc538f956ea4a78e","created_at":"2026-10-02T14:16:13.374Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.384Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-68a5d940dc538f956ea4a78e: start","job_status":"start","queue_duration_s":2.911334,"scheduling_latency_s":2.91137}1948gitlab # [ 418.466727] gitaly[924]: time="2026-10-02T14:16:16.236Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ3ZTWXNTHAG41MJQY9JD grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:16.222" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1949gitlab # [ 418.481022] gitlab-workhorse[823]: time="2026-10-02T14:16:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3351950gitlab # [ 418.483235] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 335 "" "git/2.55.0" 991951gitlab # [ 418.485300] nixbot[3547]: {"ts": "2026-10-02T14:16:16.384Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1952gitlab # [ 418.487847] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.381Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"68a5d940dc538f956ea4a78e","created_at":"2026-10-02T14:16:13.374Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.384Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-68a5d940dc538f956ea4a78e: done: 0.084933 sec","job_status":"done","queue_duration_s":2.911334,"scheduling_latency_s":2.91137,"redis_calls":13,"redis_duration_s":0.003228,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000556,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000659,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004189,"duration_s":0.084933,"completed_at":"2026-10-02T14:16:16.380Z","load_balancing_strategy":"primary","db_duration_s":0.015115,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 418.537546] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.383Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fd94fceb6951f52e18d4938d","created_at":"2026-10-02T14:16:13.385Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.394Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fd94fceb6951f52e18d4938d: start","job_status":"start","queue_duration_s":2.988722,"scheduling_latency_s":2.988761}1954gitlab # [ 418.554985] gitaly[924]: time="2026-10-02T14:16:16.236Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ3ZTWXNTHAG41MJQY9JD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:16.222" grpc.time_ms=13.983 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1955gitlab # [ 418.570290] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.449Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fd94fceb6951f52e18d4938d","created_at":"2026-10-02T14:16:13.385Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.394Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fd94fceb6951f52e18d4938d: done: 0.066377 sec","job_status":"done","queue_duration_s":2.988722,"scheduling_latency_s":2.988761,"redis_calls":10,"redis_duration_s":0.002728,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000593,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001591,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000544,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037794,"duration_s":0.066377,"completed_at":"2026-10-02T14:16:16.449Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1956gitlab # [ 418.617251] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.452Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9020b671c48815e032e5f130","created_at":"2026-10-02T14:16:13.396Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.406Z","job_size_bytes":19,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9020b671c48815e032e5f130: start","job_status":"start","queue_duration_s":3.045771,"scheduling_latency_s":3.04581}1957gitlab # [ 418.634393] gitaly[924]: time="2026-10-02T14:16:16.339Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ43GX47QW2N7AB5YTQB1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:16.320" pid=924 remote_ip="2001:db8:1::1" request_sha=33b98a471e2c2c883eba84c7dafa01dc7d888834 response_bytes=335 span.kind=server system=grpc1958gitlab # [ 418.645098] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.524Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"9020b671c48815e032e5f130","created_at":"2026-10-02T14:16:13.396Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.406Z","job_size_bytes":19,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-9020b671c48815e032e5f130: done: 0.07203 sec","job_status":"done","queue_duration_s":3.045771,"scheduling_latency_s":3.04581,"redis_calls":11,"redis_duration_s":0.006252000000000001,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001476,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003602,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00072058,"duration_s":0.07203,"completed_at":"2026-10-02T14:16:16.524Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1959gitlab # [ 418.692708] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.526Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0b56bd263853b0d6c2682248","created_at":"2026-10-02T14:16:13.410Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.419Z","job_size_bytes":66,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0b56bd263853b0d6c2682248: start","job_status":"start","queue_duration_s":3.107861,"scheduling_latency_s":3.107901}1960gitlab # [ 418.711332] gitaly[924]: time="2026-10-02T14:16:16.339Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ43GX47QW2N7AB5YTQB1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:16.320" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1961gitlab # [ 418.726536] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.595Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"0b56bd263853b0d6c2682248","created_at":"2026-10-02T14:16:13.410Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.419Z","job_size_bytes":66,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-0b56bd263853b0d6c2682248: done: 0.068253 sec","job_status":"done","queue_duration_s":3.107861,"scheduling_latency_s":3.107901,"redis_calls":10,"redis_duration_s":0.006083,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00059,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002091,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003402,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059548,"duration_s":0.068253,"completed_at":"2026-10-02T14:16:16.594Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008531,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 418.775629] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4f8114d30fb536f8ae236c5b","created_at":"2026-10-02T14:16:13.420Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.429Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4f8114d30fb536f8ae236c5b: start","job_status":"start","queue_duration_s":3.168815,"scheduling_latency_s":3.168853}1963gitlab # [ 418.793034] gitaly[924]: time="2026-10-02T14:16:16.339Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1129 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ43GX47QW2N7AB5YTQB1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:16.320" grpc.time_ms=18.856 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1964gitlab # [ 418.808765] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.684Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"4f8114d30fb536f8ae236c5b","created_at":"2026-10-02T14:16:13.420Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.429Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-4f8114d30fb536f8ae236c5b: done: 0.086356 sec","job_status":"done","queue_duration_s":3.168815,"scheduling_latency_s":3.168853,"redis_calls":10,"redis_duration_s":0.00289,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000524,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001601,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000765,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00065002,"duration_s":0.086356,"completed_at":"2026-10-02T14:16:16.684Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 418.856802] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.687Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"eee1ca34a2ab3a62be835920","created_at":"2026-10-02T14:16:13.441Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.448Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-eee1ca34a2ab3a62be835920: start","job_status":"start","queue_duration_s":3.238802,"scheduling_latency_s":3.238838}1966gitlab # [ 418.876533] nixbot[3547]: {"ts": "2026-10-02T14:16:16.707Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 \"HTTP/1.1 201 Created\""}1967gitlab # [ 418.880090] gitaly[924]: time="2026-10-02T14:16:16.494Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:46.192" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:16.493" grpc.time_ms=0.682 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1968gitlab # [ 418.894684] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3051969gitlab # [ 418.898052] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.749Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"eee1ca34a2ab3a62be835920","created_at":"2026-10-02T14:16:13.441Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.448Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-eee1ca34a2ab3a62be835920: done: 0.062218 sec","job_status":"done","queue_duration_s":3.238802,"scheduling_latency_s":3.238838,"redis_calls":10,"redis_duration_s":0.0030689999999999997,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000478,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001999,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042192,"duration_s":0.062218,"completed_at":"2026-10-02T14:16:16.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1970gitlab # [ 418.949051] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.751Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"08caf4ea95db137d3b5640ff","created_at":"2026-10-02T14:16:13.449Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.458Z","job_size_bytes":6,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-08caf4ea95db137d3b5640ff: start","job_status":"start","queue_duration_s":3.292773,"scheduling_latency_s":3.292807}1971gitlab # [ 418.967035] gitaly[924]: time="2026-10-02T14:16:16.621Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:26.081" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:16.612" grpc.time_ms=9.164 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 418.982480] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.798Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"08caf4ea95db137d3b5640ff","created_at":"2026-10-02T14:16:13.449Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.458Z","job_size_bytes":6,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-08caf4ea95db137d3b5640ff: done: 0.046683 sec","job_status":"done","queue_duration_s":3.292773,"scheduling_latency_s":3.292807,"redis_calls":10,"redis_duration_s":0.004031,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000554,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001673,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00094186,"duration_s":0.046683,"completed_at":"2026-10-02T14:16:16.797Z","load_balancing_strategy":"primary","db_duration_s":0.00503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973gitlab # [ 419.030713] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.800Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"133850405a7e2eaaadc981f4","created_at":"2026-10-02T14:16:13.459Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.467Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-133850405a7e2eaaadc981f4: start","job_status":"start","queue_duration_s":3.332758,"scheduling_latency_s":3.332794}1974gitlab # [ 419.047922] gitaly[924]: time="2026-10-02T14:16:16.803Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ4J7282W0WN6GZ4NYJPM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:46.501" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:16.801" grpc.time_ms=1.209 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'1976gitlab # [ 419.062664] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.872Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"133850405a7e2eaaadc981f4","created_at":"2026-10-02T14:16:13.459Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.467Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-133850405a7e2eaaadc981f4: done: 0.072092 sec","job_status":"done","queue_duration_s":3.332758,"scheduling_latency_s":3.332794,"redis_calls":10,"redis_duration_s":0.0028510000000000002,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000594,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047333,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046874,"duration_s":0.072092,"completed_at":"2026-10-02T14:16:16.872Z","load_balancing_strategy":"primary","db_duration_s":0.016048,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1977gitlab # [ 419.110333] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:16.874Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"d0ec39593590fc65e9fe5847","created_at":"2026-10-02T14:16:13.476Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.485Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-d0ec39593590fc65e9fe5847: start","job_status":"start","queue_duration_s":3.388795,"scheduling_latency_s":3.388833}1978gitlab # [ 419.127965] gitaly[924]: time="2026-10-02T14:16:16.887Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ4J7282W0WN6GZ4NYJPM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:26.055" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:16.876" grpc.time_ms=11.239 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1979gitlab # [ 419.142900] nixbot[3547]: {"ts": "2026-10-02T14:16:16.988Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 \"HTTP/1.1 201 Created\""}1980gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.09 seconds)1981gitlab # [ 419.146220] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:17.002Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"d0ec39593590fc65e9fe5847","created_at":"2026-10-02T14:16:13.476Z","trace_propagation_headers":{"sentry-trace":"20536b795e4a4c6c9d34446f3c3b03f3-06fd999d59974442","baggage":"sentry-trace_id=20536b795e4a4c6c9d34446f3c3b03f3,sentry-release=Unknown"},"correlation_id":"01M3YFFGW27D61DW6YVDZ7WFB8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.485Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-d0ec39593590fc65e9fe5847: done: 0.128228 sec","job_status":"done","queue_duration_s":3.388795,"scheduling_latency_s":3.388833,"redis_calls":14,"redis_duration_s":0.004828,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000728,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001645,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002455,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088817,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047918,"duration_s":0.128228,"completed_at":"2026-10-02T14:16:17.002Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1982gitlab # [ 419.195620] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:17.005Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cad3794e2028f252e2b05888","created_at":"2026-10-02T14:16:13.726Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-c0a8dd867b5f4ea0","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"b91a82e56d323ca1acc908a7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.730Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cad3794e2028f252e2b05888: start","job_status":"start","queue_duration_s":3.274605,"scheduling_latency_s":3.274644}1983gitlab # [ 419.216109] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2741984gitlab # [ 419.218834] gitaly[924]: time="2026-10-02T14:16:17.050Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ diskcache=e9ab4ac6-4c37-42d0-b5ed-9ba735954ebe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:16:17.041" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.041" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985gitlab # [ 419.229596] gitaly[924]: time="2026-10-02T14:16:17.051Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T20:16:17.041" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.041" grpc.time_ms=9.25 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 419.243850] nixbot[3547]: INFO: ::ffff:127.0.0.1:54530 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK1987gitlab # [ 419.246053] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:17.079Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"cad3794e2028f252e2b05888","created_at":"2026-10-02T14:16:13.726Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-c0a8dd867b5f4ea0","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"b91a82e56d323ca1acc908a7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.730Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-cad3794e2028f252e2b05888: done: 0.075027 sec","job_status":"done","queue_duration_s":3.274605,"scheduling_latency_s":3.274644,"gitaly_calls":1,"gitaly_duration_s":0.012707,"redis_calls":12,"redis_duration_s":0.00321,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001215,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000724,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001271,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04298704,"duration_s":0.075027,"completed_at":"2026-10-02T14:16:17.079Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008367,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1988gitlab # [ 419.295079] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:17.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"28d69074e7e1bce56d81aaae","created_at":"2026-10-02T14:16:13.926Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-c0a8dd867b5f4ea0","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:03bcbf36384ef0138dc15cf4a1baac507d87cb3fff8fe8f27ba6a05b57fcfd38","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.932Z","job_size_bytes":467,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-28d69074e7e1bce56d81aaae: start","job_status":"start","queue_duration_s":3.150262,"scheduling_latency_s":3.150302}1989gitlab # [ 419.314130] gitaly[924]: time="2026-10-02T14:16:17.088Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ4V3T0P6HFNHY7TCC8Z3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:47.087" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.088" grpc.time_ms=0.71 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 419.327865] gitaly[924]: time="2026-10-02T14:16:17.120Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:47.018" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.119" grpc.time_ms=0.797 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1991gitlab # [ 419.341360] gitaly[924]: time="2026-10-02T14:16:17.134Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:16:27.003" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:17.124" grpc.time_ms=9.053 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1992gitlab # [ 419.355830] nixbot[3547]: {"ts": "2026-10-02T14:16:17.242Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 \"HTTP/1.1 201 Created\""}1993gitlab # [ 419.359492] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/88b61d9ace3e1870aed365de3bf04ea0e6962e72 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2451994gitlab # [ 419.362414] gitaly[924]: time="2026-10-02T14:16:17.157Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ4V3T0P6HFNHY7TCC8Z3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:27.117" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:17.148" grpc.time_ms=9.361 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1995gitlab # [ 419.377332] gitaly[924]: time="2026-10-02T14:16:17.193Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:16:47.092" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.192" grpc.time_ms=0.935 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1996gitlab # [ 419.391368] gitaly[924]: time="2026-10-02T14:16:17.203Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=297 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T14:16:47.097" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.198" grpc.time_ms=5.397 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1997gitlab # [ 419.409695] gitaly[924]: time="2026-10-02T14:16:17.250Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:27.120" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:17.241" grpc.time_ms=8.959 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998gitlab # [ 419.453872] gitaly[924]: time="2026-10-02T14:16:17.361Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ53WWR46H0BFJTTWW91C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.350" grpc.time_ms=10.472 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc1999gitlab # [ 419.470428] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 852000gitlab # [ 419.572232] gitaly[924]: time="2026-10-02T14:16:17.479Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ56M3C8K86V3GYHFZPP4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.466" pid=924 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2001gitlab # [ 419.589891] gitaly[924]: time="2026-10-02T14:16:17.479Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ56M3C8K86V3GYHFZPP4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.466" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2002gitlab # [ 419.611751] gitlab-workhorse[823]: time="2026-10-02T14:16:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672003gitlab # [ 419.614700] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1162004gitlab # [ 419.617565] gitaly[924]: time="2026-10-02T14:16:17.480Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ56M3C8K86V3GYHFZPP4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:17.466" grpc.time_ms=13.476 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2005gitlab # [ 420.088104] gitaly[924]: time="2026-10-02T14:16:17.994Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T14:16:47.877" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.977" grpc.time_ms=16.89 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2006gitlab # [ 420.113292] gitaly[924]: time="2026-10-02T14:16:18.013Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFHWPP5VSH6F9QRX1XH4CZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T14:16:47.898" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:17.998" grpc.time_ms=14.562 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2007gitlab # [ 420.136129] sidekiq-cluster[3422]: {"severity":"WARN","time":"2026-10-02T14:16:18.032Z","class":"CreatePipelineWorker","correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"88b61d9ace3e1870aed365de3bf04ea0e6962e72","after":"45e21e5de49e68885eab25da3122e70bb390a5df","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"45e21e5de49e68885eab25da3122e70bb390a5df","ref":"refs/heads/master"},"retry":0}2008gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'2009gitlab # [ 420.147622] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"28d69074e7e1bce56d81aaae","created_at":"2026-10-02T14:16:13.926Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-c0a8dd867b5f4ea0","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:03bcbf36384ef0138dc15cf4a1baac507d87cb3fff8fe8f27ba6a05b57fcfd38","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.932Z","job_size_bytes":467,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-28d69074e7e1bce56d81aaae: done: 0.958177 sec","job_status":"done","queue_duration_s":3.150262,"scheduling_latency_s":3.150302,"gitaly_calls":7,"gitaly_duration_s":0.078365,"redis_calls":33,"redis_duration_s":0.008714,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.002774,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003449,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00122,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.855997,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.958177,"completed_at":"2026-10-02T14:16:18.040Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025484,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2010gitlab # [ 420.210673] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.043Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"491877db-2fda-4f55-9b26-e1f9d31d5f33\"}"],"class":"WebHookWorker","jid":"efa02b5112dd07282dfcab2f","created_at":"2026-10-02T14:16:13.962Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-c0a8dd867b5f4ea0","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ffa71183ad447f0b2d146df94badf4ffde214a70cc06068e7cae67ab95ab0162","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.966Z","job_size_bytes":1749,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-efa02b5112dd07282dfcab2f: start","job_status":"start","queue_duration_s":4.07653,"scheduling_latency_s":4.076566}2011gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.07 seconds)2012gitlab # [ 420.230836] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.109Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"491877db-2fda-4f55-9b26-e1f9d31d5f33\"}"],"class":"WebHookWorker","jid":"efa02b5112dd07282dfcab2f","created_at":"2026-10-02T14:16:13.962Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-c0a8dd867b5f4ea0","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ffa71183ad447f0b2d146df94badf4ffde214a70cc06068e7cae67ab95ab0162","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:13.966Z","job_size_bytes":1749,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-efa02b5112dd07282dfcab2f: done: 0.066457 sec","job_status":"done","queue_duration_s":4.07653,"scheduling_latency_s":4.076566,"redis_calls":14,"redis_duration_s":0.006580000000000001,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000588,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004682,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.008899559999974826,"cpu_s":0.043116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.066457,"completed_at":"2026-10-02T14:16:18.109Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2013gitlab # [ 420.282687] nixbot[3547]: INFO: ::ffff:127.0.0.1:54540 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2014gitlab # [ 420.284080] nixbot[3547]: INFO: ::ffff:127.0.0.1:54546 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK2015gitlab # [ 420.286123] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.112Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"b259f4c3a23a0bbb3e5c12d6","created_at":"2026-10-02T14:16:14.172Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-044a90aec5334d64","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:14.181Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-b259f4c3a23a0bbb3e5c12d6: start","job_status":"start","queue_duration_s":3.930668,"scheduling_latency_s":3.930705}2016gitlab # [ 420.303363] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.208Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"b259f4c3a23a0bbb3e5c12d6","created_at":"2026-10-02T14:16:14.172Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-044a90aec5334d64","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:14.181Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-b259f4c3a23a0bbb3e5c12d6: done: 0.096006 sec","job_status":"done","queue_duration_s":3.930668,"scheduling_latency_s":3.930705,"redis_calls":19,"redis_duration_s":0.009155,"redis_read_bytes":21,"redis_write_bytes":4289,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000988,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000599,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2260,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.006543,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0116724,"duration_s":0.096006,"completed_at":"2026-10-02T14:16:18.208Z","load_balancing_strategy":"primary","db_duration_s":0.010828,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2017gitlab # [ 420.352056] gitaly[924]: time="2026-10-02T14:16:18.220Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ5Y6BQ5FEPX0EWRK079K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.208" grpc.time_ms=11.907 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2018gitlab # [ 420.367206] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1022019gitlab # [ 420.369679] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.210Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","84aaafd9860e1956274efe5d"],"class":"WebHooks::LogExecutionWorker","jid":"c4aecc97bce073b9c6fb39ca","created_at":"2026-10-02T14:16:15.984Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-5eb1087807144d59","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:17b0ea3a8598b7dd9417ae0946f5156c6bd511e7083ab2a2db57ee992659b9d7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:15.993Z","job_size_bytes":2593,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c4aecc97bce073b9c6fb39ca: start","job_status":"start","queue_duration_s":2.21683,"scheduling_latency_s":2.216863}2020gitlab # [ 420.387646] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.265Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","84aaafd9860e1956274efe5d"],"class":"WebHooks::LogExecutionWorker","jid":"c4aecc97bce073b9c6fb39ca","created_at":"2026-10-02T14:16:15.984Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-5eb1087807144d59","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:17b0ea3a8598b7dd9417ae0946f5156c6bd511e7083ab2a2db57ee992659b9d7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:15.993Z","job_size_bytes":2593,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-c4aecc97bce073b9c6fb39ca: done: 0.055205 sec","job_status":"done","queue_duration_s":2.21683,"scheduling_latency_s":2.216863,"redis_calls":10,"redis_duration_s":0.00234,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000549,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001295,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003097,"duration_s":0.055205,"completed_at":"2026-10-02T14:16:18.265Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2021gitlab # [ 420.434971] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.267Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0ffe25aef5f9efdf9ca75140","created_at":"2026-10-02T14:16:16.672Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-844bd291844240d7","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:16.679Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0ffe25aef5f9efdf9ca75140: start","job_status":"start","queue_duration_s":1.587826,"scheduling_latency_s":1.58786}2022gitlab # [ 420.451900] gitlab-workhorse[823]: time="2026-10-02T14:16:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672023gitlab # [ 420.453945] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1002024gitlab # [ 420.455911] gitaly[924]: time="2026-10-02T14:16:18.322Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ61E5GGSXZJHA2T491N3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.312" pid=924 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2025gitlab # [ 420.466731] gitaly[924]: time="2026-10-02T14:16:18.322Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ61E5GGSXZJHA2T491N3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.312" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2026gitlab # [ 420.480889] gitaly[924]: time="2026-10-02T14:16:18.323Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ61E5GGSXZJHA2T491N3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.312" grpc.time_ms=10.467 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2027gitlab # [ 420.496207] gitaly[924]: time="2026-10-02T14:16:18.393Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:18.392" grpc.time_ms=0.529 pid=924 span.kind=server system=grpc2028gitlab # [ 420.514309] gitaly[924]: time="2026-10-02T14:16:18.421Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:18.421" grpc.time_ms=0.526 pid=924 span.kind=server system=grpc2029gitlab # [ 420.533688] gitaly[924]: time="2026-10-02T14:16:18.441Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ65S2JQVDE5QNG3XA4KD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.432" grpc.time_ms=8.485 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2030gitlab # [ 420.552698] gitaly[924]: time="2026-10-02T14:16:18.449Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:18.448" grpc.time_ms=0.417 pid=924 span.kind=server system=grpc2031gitlab # [ 420.564287] gitaly[924]: time="2026-10-02T14:16:18.455Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD diskcache=2ecc0480-23ab-4898-8b31-49b3eaabdc6c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:16:48.259" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:18.360" pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032gitlab # [ 420.575487] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:18 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 792033gitlab # [ 420.578050] gitaly[924]: time="2026-10-02T14:16:18.455Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=93 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6919 command.oublock=0 command.real_time_ms=91 command.spawn_token_fork_ms=1 command.system_time_ms=38 command.user_time_ms=55 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T14:16:48.259" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:18.360" grpc.time_ms=95.799 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2034gitlab # [ 420.607572] gitaly[924]: time="2026-10-02T14:16:18.514Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ68BQJ2A4MD4D0SCG2CE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.497" pid=924 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2035gitlab # [ 420.620355] gitaly[924]: time="2026-10-02T14:16:18.515Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ68BQJ2A4MD4D0SCG2CE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.497" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2036gitlab # [ 420.635896] gitlab-workhorse[823]: time="2026-10-02T14:16:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162037gitlab # [ 420.638112] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:18 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 722038gitlab # [ 420.640197] gitaly[924]: time="2026-10-02T14:16:18.516Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ68BQJ2A4MD4D0SCG2CE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T14:16:18.497" grpc.time_ms=17.851 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc2039gitlab # [ 420.656064] gitaly[924]: time="2026-10-02T14:16:18.542Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:16:48.440" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:18.541" grpc.time_ms=1.151 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2040gitlab # [ 420.677468] gitaly[924]: time="2026-10-02T14:16:18.584Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ48F7E24PMFMQP7TWZFD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:48.483" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:18.583" grpc.time_ms=0.824 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 420.690659] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.596Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0ffe25aef5f9efdf9ca75140","created_at":"2026-10-02T14:16:16.672Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-844bd291844240d7","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:16.679Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3422,"job_status":"rescheduled","message":"PipelineProcessWorker JID-0ffe25aef5f9efdf9ca75140: rescheduled"}2042gitlab # [ 420.706427] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.606Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0ffe25aef5f9efdf9ca75140","created_at":"2026-10-02T14:16:16.672Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-844bd291844240d7","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:16.679Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0ffe25aef5f9efdf9ca75140: done: 0.338601 sec","job_status":"done","queue_duration_s":1.587826,"scheduling_latency_s":1.58786,"gitaly_calls":3,"gitaly_duration_s":0.10593,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.025989,"redis_read_bytes":148,"redis_write_bytes":18013,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.002085,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002506,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00499,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.01127,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000306,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000944,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.001803,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.180969,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.28980242,"duration_s":0.338601,"completed_at":"2026-10-02T14:16:18.606Z","load_balancing_strategy":"primary","db_duration_s":0.036346,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2043gitlab # [ 420.761625] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.608Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","efa02b5112dd07282dfcab2f"],"class":"WebHooks::LogExecutionWorker","jid":"fcbfa1c52b588ca1d9ff1b5c","created_at":"2026-10-02T14:16:18.089Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-40eaffbdffbb4ace","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:43ac93e7363b8e6410f443faf9703cf3773e9813a0fee7d9201f1528f9efe205","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.094Z","job_size_bytes":2591,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-fcbfa1c52b588ca1d9ff1b5c: start","job_status":"start","queue_duration_s":0.513337,"scheduling_latency_s":0.513368}2044gitlab # [ 420.780189] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.641Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","efa02b5112dd07282dfcab2f"],"class":"WebHooks::LogExecutionWorker","jid":"fcbfa1c52b588ca1d9ff1b5c","created_at":"2026-10-02T14:16:18.089Z","trace_propagation_headers":{"sentry-trace":"bcb9e1e93e594aa1b015a0defff6618b-40eaffbdffbb4ace","baggage":"sentry-trace_id=bcb9e1e93e594aa1b015a0defff6618b,sentry-release=Unknown"},"correlation_id":"01M3YFHWPP5VSH6F9QRX1XH4CZ","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:43ac93e7363b8e6410f443faf9703cf3773e9813a0fee7d9201f1528f9efe205","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.094Z","job_size_bytes":2591,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-fcbfa1c52b588ca1d9ff1b5c: done: 0.033373 sec","job_status":"done","queue_duration_s":0.513337,"scheduling_latency_s":0.513368,"redis_calls":7,"redis_duration_s":0.002205,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000446,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022864,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033373,"completed_at":"2026-10-02T14:16:18.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003878,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2045gitlab # [ 420.826279] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.643Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f3754053ee0057685cbc98bf","created_at":"2026-10-02T14:16:18.188Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-5e36625526e24033","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.196Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f3754053ee0057685cbc98bf: start","job_status":"start","queue_duration_s":0.446334,"scheduling_latency_s":0.446364}2046gitlab # [ 420.844761] gitaly[924]: time="2026-10-02T14:16:18.621Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:48.419" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:18.620" grpc.time_ms=0.8 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047gitlab # [ 420.858909] gitaly[924]: time="2026-10-02T14:16:18.644Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T14:16:48.432" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:18.632" grpc.time_ms=11.39 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2048gitlab # [ 420.873958] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.757Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f3754053ee0057685cbc98bf","created_at":"2026-10-02T14:16:18.188Z","trace_propagation_headers":{"sentry-trace":"83a25635ec2b4a3c9f7f8875ca5d3c39-5e36625526e24033","baggage":"sentry-trace_id=83a25635ec2b4a3c9f7f8875ca5d3c39,sentry-release=Unknown"},"correlation_id":"01M3YFFAHW9TRM88X23S3NB33S","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.196Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f3754053ee0057685cbc98bf: done: 0.114605 sec","job_status":"done","queue_duration_s":0.446334,"scheduling_latency_s":0.446364,"redis_calls":11,"redis_duration_s":0.003503,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000419,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001459,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001625,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.082114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045044,"duration_s":0.114605,"completed_at":"2026-10-02T14:16:18.757Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024506,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2049gitlab # [ 420.921513] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.759Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"48b0069b9195a54c4b7467fe","created_at":"2026-10-02T14:16:18.348Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.352Z","job_size_bytes":23,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-48b0069b9195a54c4b7467fe: start","job_status":"start","queue_duration_s":0.407547,"scheduling_latency_s":0.407578}2050gitlab # [ 420.939792] gitaly[924]: time="2026-10-02T14:16:18.661Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:28.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:18.652" grpc.time_ms=9.03 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2051gitlab # [ 420.954607] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.812Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"48b0069b9195a54c4b7467fe","created_at":"2026-10-02T14:16:18.348Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.352Z","job_size_bytes":23,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-48b0069b9195a54c4b7467fe: done: 0.053139 sec","job_status":"done","queue_duration_s":0.407547,"scheduling_latency_s":0.407578,"redis_calls":7,"redis_duration_s":0.001917,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053139,"completed_at":"2026-10-02T14:16:18.812Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009023,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2052gitlab # [ 421.000149] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.814Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9f1f63ef029d95e9b2c21e39","created_at":"2026-10-02T14:16:18.462Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.469Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9f1f63ef029d95e9b2c21e39: start","job_status":"start","queue_duration_s":0.344814,"scheduling_latency_s":0.344843}2053gitlab # [ 421.017912] gitaly[924]: time="2026-10-02T14:16:18.682Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T14:16:28.031" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:18.681" grpc.time_ms=0.386 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2054gitlab # [ 421.029712] nixbot[3547]: {"ts": "2026-10-02T14:16:18.914Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df \"HTTP/1.1 201 Created\""}2055gitlab # [ 421.032591] nixbot[3547]: {"ts": "2026-10-02T14:16:18.924Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-45e21e5de49e-5"}2056gitlab # [ 421.034962] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.845Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9f1f63ef029d95e9b2c21e39","created_at":"2026-10-02T14:16:18.462Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.469Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9f1f63ef029d95e9b2c21e39: done: 0.031043 sec","job_status":"done","queue_duration_s":0.344814,"scheduling_latency_s":0.344843,"redis_calls":7,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001113,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031043,"completed_at":"2026-10-02T14:16:18.845Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005099,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2057gitlab # [ 421.080170] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.847Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"df71526bd91dd3f647493cfc","created_at":"2026-10-02T14:16:18.484Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.489Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-df71526bd91dd3f647493cfc: start","job_status":"start","queue_duration_s":0.357886,"scheduling_latency_s":0.357913}2058gitlab # [ 421.096750] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:18 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 3662059gitlab # [ 421.099740] gitaly[924]: time="2026-10-02T14:16:18.697Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T14:16:28.038" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:18.688" grpc.time_ms=8.301 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2060gitlab # [ 421.114665] gitaly[924]: time="2026-10-02T14:16:18.747Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:18.746" grpc.time_ms=0.868 pid=924 span.kind=server system=grpc2061gitlab # [ 421.125241] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.884Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"df71526bd91dd3f647493cfc","created_at":"2026-10-02T14:16:18.484Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.489Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-df71526bd91dd3f647493cfc: done: 0.036886 sec","job_status":"done","queue_duration_s":0.357886,"scheduling_latency_s":0.357913,"redis_calls":7,"redis_duration_s":0.001277,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036886,"completed_at":"2026-10-02T14:16:18.884Z","load_balancing_strategy":"primary","db_duration_s":0.007075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2062gitlab # [ 421.170700] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.887Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6f44d5f0267a12bf5e587a2b","created_at":"2026-10-02T14:16:18.501Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.504Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6f44d5f0267a12bf5e587a2b: start","job_status":"start","queue_duration_s":0.382592,"scheduling_latency_s":0.382621}2063gitlab # [ 421.187507] gitaly[924]: time="2026-10-02T14:16:18.776Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T14:16:18.776" grpc.time_ms=0.475 pid=924 span.kind=server system=grpc2064gitlab # [ 421.197743] gitaly[924]: time="2026-10-02T14:16:18.787Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J diskcache=fee8e15f-12f7-41e7-a59c-79f92403cd6b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:16:28.049" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:18.700" pid=924 ref=refs/keep-around/45e21e5de49e68885eab25da3122e70bb390a5df remote_ip="2001:db8:1::1" revision=45e21e5de49e68885eab25da3122e70bb390a5df span.kind=server system=grpc username=root2065gitlab # [ 421.209807] nixbot[3547]: {"ts": "2026-10-02T14:16:19.082Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-45e21e5de49e-5", "argv0": "bwrap"}2066gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df'2067gitlab # [ 421.212303] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.937Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"6f44d5f0267a12bf5e587a2b","created_at":"2026-10-02T14:16:18.501Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.504Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-6f44d5f0267a12bf5e587a2b: done: 0.049979 sec","job_status":"done","queue_duration_s":0.382592,"scheduling_latency_s":0.382621,"redis_calls":10,"redis_duration_s":0.002577,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000619,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049979,"completed_at":"2026-10-02T14:16:18.937Z","load_balancing_strategy":"primary","db_duration_s":0.009793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2068gitlab # [ 421.256840] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.939Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"988ebe6d2ace1c26b079aeae","created_at":"2026-10-02T14:16:18.505Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.508Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-988ebe6d2ace1c26b079aeae: start","job_status":"start","queue_duration_s":0.430986,"scheduling_latency_s":0.43102}2069gitlab # [ 421.273950] gitaly[924]: time="2026-10-02T14:16:18.788Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5348 command.oublock=8 command.real_time_ms=79 command.spawn_token_fork_ms=2 command.system_time_ms=49 command.user_time_ms=25 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T14:16:28.049" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T14:16:18.700" grpc.time_ms=87.902 pid=924 ref=refs/keep-around/45e21e5de49e68885eab25da3122e70bb390a5df remote_ip="2001:db8:1::1" revision=45e21e5de49e68885eab25da3122e70bb390a5df span.kind=server system=grpc username=root2070gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df', in 0.07 seconds)2071(finished: subtest: push delivers the webhook, in 88.76 seconds)2072subtest: eval and build post commit statuses2073gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2074gitlab # [ 421.289597] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.980Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"988ebe6d2ace1c26b079aeae","created_at":"2026-10-02T14:16:18.505Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.508Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-988ebe6d2ace1c26b079aeae: done: 0.041149 sec","job_status":"done","queue_duration_s":0.430986,"scheduling_latency_s":0.43102,"redis_calls":7,"redis_duration_s":0.00151,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000394,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041149,"completed_at":"2026-10-02T14:16:18.980Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2075gitlab # [ 421.334760] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:18.982Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"dcbde65bb71b5d8daf24ce3b","created_at":"2026-10-02T14:16:18.509Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.512Z","job_size_bytes":19,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-dcbde65bb71b5d8daf24ce3b: start","job_status":"start","queue_duration_s":0.469266,"scheduling_latency_s":0.469285}2076gitlab # [ 421.352926] gitaly[924]: time="2026-10-02T14:16:18.792Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:48.591" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:18.792" grpc.time_ms=0.657 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2077gitlab # [ 421.366492] nixbot[3547]: INFO: ::ffff:127.0.0.1:59614 - "GET /api/repos/gitlab/root/test-flake/builds?commit=45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 200 OK2078gitlab # [ 421.368667] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.012Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"dcbde65bb71b5d8daf24ce3b","created_at":"2026-10-02T14:16:18.509Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.512Z","job_size_bytes":19,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-dcbde65bb71b5d8daf24ce3b: done: 0.030545 sec","job_status":"done","queue_duration_s":0.469266,"scheduling_latency_s":0.469285,"redis_calls":8,"redis_duration_s":0.001175,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030545,"completed_at":"2026-10-02T14:16:19.012Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003981,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2079gitlab # [ 421.415028] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.013Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"26b1a11184f8bbc89c0d635c","created_at":"2026-10-02T14:16:18.514Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.517Z","job_size_bytes":66,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-26b1a11184f8bbc89c0d635c: start","job_status":"start","queue_duration_s":0.496769,"scheduling_latency_s":0.496787}2080gitlab # [ 421.433840] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.041Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"26b1a11184f8bbc89c0d635c","created_at":"2026-10-02T14:16:18.514Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.517Z","job_size_bytes":66,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-26b1a11184f8bbc89c0d635c: done: 0.027137 sec","job_status":"done","queue_duration_s":0.496769,"scheduling_latency_s":0.496787,"redis_calls":7,"redis_duration_s":0.001025,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000284,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027137,"completed_at":"2026-10-02T14:16:19.041Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002708,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 421.479598] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.042Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cba108dc7994e970b7cd3ee5","created_at":"2026-10-02T14:16:18.517Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.523Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cba108dc7994e970b7cd3ee5: start","job_status":"start","queue_duration_s":0.518477,"scheduling_latency_s":0.518494}2082gitlab # [ 421.497282] nix-daemon[3710]: accepted connection from pid 4774, user nixbot2083gitlab # [ 421.498296] gitaly[924]: time="2026-10-02T14:16:19.368Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T14:16:49.267" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:19.367" grpc.time_ms=0.762 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2084gitlab # [ 421.511499] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.069Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"cba108dc7994e970b7cd3ee5","created_at":"2026-10-02T14:16:18.517Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.523Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-cba108dc7994e970b7cd3ee5: done: 0.02763 sec","job_status":"done","queue_duration_s":0.518477,"scheduling_latency_s":0.518494,"redis_calls":7,"redis_duration_s":0.001008,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02763,"completed_at":"2026-10-02T14:16:19.069Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 421.557915] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.071Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3b2563755a704ba6404e0fcb","created_at":"2026-10-02T14:16:18.524Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.526Z","job_size_bytes":6,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3b2563755a704ba6404e0fcb: start","job_status":"start","queue_duration_s":0.544906,"scheduling_latency_s":0.544925}2086gitlab # [ 421.574563] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.086Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3b2563755a704ba6404e0fcb","created_at":"2026-10-02T14:16:18.524Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.526Z","job_size_bytes":6,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3b2563755a704ba6404e0fcb: done: 0.015134 sec","job_status":"done","queue_duration_s":0.544906,"scheduling_latency_s":0.544925,"redis_calls":7,"redis_duration_s":0.001006,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015134,"completed_at":"2026-10-02T14:16:19.086Z","load_balancing_strategy":"primary","db_duration_s":0.000348,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 421.622210] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.087Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"99337e0daf6c29963cf27f5c","created_at":"2026-10-02T14:16:18.526Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.528Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-99337e0daf6c29963cf27f5c: start","job_status":"start","queue_duration_s":0.559285,"scheduling_latency_s":0.559301}2088gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.35 seconds)2089gitlab # [ 421.639477] gitaly[924]: time="2026-10-02T14:16:19.419Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ6BJXE46Z8JRW2E0J65J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:49.317" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:19.417" grpc.time_ms=0.981 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2090gitlab # [ 421.653234] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.111Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"99337e0daf6c29963cf27f5c","created_at":"2026-10-02T14:16:18.526Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.528Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-99337e0daf6c29963cf27f5c: done: 0.024236 sec","job_status":"done","queue_duration_s":0.559285,"scheduling_latency_s":0.559301,"redis_calls":7,"redis_duration_s":0.001174,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024236,"completed_at":"2026-10-02T14:16:19.111Z","load_balancing_strategy":"primary","db_duration_s":0.002931,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2091gitlab # [ 421.697548] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.113Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7f3e594acf0f05c0ff1de815","created_at":"2026-10-02T14:16:18.597Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.600Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7f3e594acf0f05c0ff1de815: start","job_status":"start","queue_duration_s":0.51314,"scheduling_latency_s":0.513157}2092gitlab # [ 421.714651] gitaly[924]: time="2026-10-02T14:16:19.509Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ71K7AF74136YQ9X2QR5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:49.306" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:19.506" grpc.time_ms=2.241 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2093gitlab # [ 421.729015] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 3042094gitlab # [ 421.731863] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.141Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"7f3e594acf0f05c0ff1de815","created_at":"2026-10-02T14:16:18.597Z","trace_propagation_headers":{"sentry-trace":"c8381d5e305040abb6680285995564bb-810d052f0e824a36","baggage":"sentry-trace_id=c8381d5e305040abb6680285995564bb,sentry-release=Unknown"},"correlation_id":"01M3YFJ48F7E24PMFMQP7TWZFD","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.600Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7f3e594acf0f05c0ff1de815: done: 0.027985 sec","job_status":"done","queue_duration_s":0.51314,"scheduling_latency_s":0.513157,"redis_calls":8,"redis_duration_s":0.001563,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001222,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000341,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027985,"completed_at":"2026-10-02T14:16:19.141Z","load_balancing_strategy":"primary","db_duration_s":0.004318,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2095gitlab # [ 421.778941] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.142Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"add4cd840d46ee72b15c85a3","created_at":"2026-10-02T14:16:18.794Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-946b891f13a940c7","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.798Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-add4cd840d46ee72b15c85a3: start","job_status":"start","queue_duration_s":0.343557,"scheduling_latency_s":0.343574}2096gitlab # [ 421.796918] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.179Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"add4cd840d46ee72b15c85a3","created_at":"2026-10-02T14:16:18.794Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-946b891f13a940c7","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.798Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-add4cd840d46ee72b15c85a3: done: 0.037252 sec","job_status":"done","queue_duration_s":0.343557,"scheduling_latency_s":0.343574,"redis_calls":7,"redis_duration_s":0.003226,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000438,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037252,"completed_at":"2026-10-02T14:16:19.179Z","load_balancing_strategy":"primary","db_duration_s":0.006018,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2097gitlab # [ 421.842028] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.181Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2ae75325ec55f9875f35356e","created_at":"2026-10-02T14:16:18.799Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-946b891f13a940c7","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.803Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2ae75325ec55f9875f35356e: start","job_status":"start","queue_duration_s":0.378119,"scheduling_latency_s":0.378145}2098gitlab # [ 421.860701] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.265Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2ae75325ec55f9875f35356e","created_at":"2026-10-02T14:16:18.799Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-946b891f13a940c7","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.803Z","job_size_bytes":65,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2ae75325ec55f9875f35356e: done: 0.083529 sec","job_status":"done","queue_duration_s":0.378119,"scheduling_latency_s":0.378145,"redis_calls":10,"redis_duration_s":0.0045780000000000005,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.003343,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.083529,"completed_at":"2026-10-02T14:16:19.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029616,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 421.907243] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.267Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"23e64d2c40d45a728bfffc69","created_at":"2026-10-02T14:16:18.890Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-946b891f13a940c7","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.895Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-23e64d2c40d45a728bfffc69: start","job_status":"start","queue_duration_s":0.371585,"scheduling_latency_s":0.371615}2100gitlab # [ 421.924249] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.440Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"23e64d2c40d45a728bfffc69","created_at":"2026-10-02T14:16:18.890Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-946b891f13a940c7","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:18.895Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-23e64d2c40d45a728bfffc69: done: 0.173371 sec","job_status":"done","queue_duration_s":0.371585,"scheduling_latency_s":0.371615,"gitaly_calls":2,"gitaly_duration_s":0.007582,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.022533,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.003645,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001823,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000589,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004728,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000595,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.005574,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.122054,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12389166,"duration_s":0.173371,"completed_at":"2026-10-02T14:16:19.440Z","load_balancing_strategy":"primary","db_duration_s":0.031428,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2101gitlab # [ 421.979606] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.443Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"dd65c2856df71718b2097e8c","created_at":"2026-10-02T14:16:19.347Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-4d20ef62f45b4458","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:19.352Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-dd65c2856df71718b2097e8c: start","job_status":"start","queue_duration_s":0.091105,"scheduling_latency_s":0.091135}2102gitlab # [ 421.996944] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:19.649Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"dd65c2856df71718b2097e8c","created_at":"2026-10-02T14:16:19.347Z","trace_propagation_headers":{"sentry-trace":"d2132c085b7c49b7a63c5c982bf7b1c3-4d20ef62f45b4458","baggage":"sentry-trace_id=d2132c085b7c49b7a63c5c982bf7b1c3,sentry-release=Unknown"},"correlation_id":"01M3YFJ6BJXE46Z8JRW2E0J65J","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:19.352Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-dd65c2856df71718b2097e8c: done: 0.206671 sec","job_status":"done","queue_duration_s":0.091105,"scheduling_latency_s":0.091135,"redis_calls":7,"redis_duration_s":0.0019329999999999998,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.188323,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.206671,"completed_at":"2026-10-02T14:16:19.649Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2103gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2104gitlab # [ 422.796067] gitaly[924]: time="2026-10-02T14:16:20.702Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ8BYRP936387ZN3TBXH7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:50.500" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:20.701" grpc.time_ms=1.022 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2105gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.19 seconds)2106gitlab # [ 422.843732] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1432107gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2108gitlab # [ 423.979906] gitaly[924]: time="2026-10-02T14:16:21.886Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJ9GW223RY5MMZZM9CNV9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:51.684" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:21.885" grpc.time_ms=0.922 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2109gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.19 seconds)2110gitlab # [ 424.030928] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:21 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1482111gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2112gitlab # [ 425.167641] gitaly[924]: time="2026-10-02T14:16:23.074Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJAP03KPSJ56MMZC4NB6C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:53.072" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:23.073" grpc.time_ms=0.989 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.18 seconds)2114gitlab # [ 425.215885] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1452115gitlab # [ 426.046083] nix-daemon[3710]: accepted connection from pid 4808, user nixbot2116gitlab # [ 426.061489] nix-daemon[3710]: accepted connection from pid 4774, user nixbot2117gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2118gitlab # [ 426.352321] gitaly[924]: time="2026-10-02T14:16:24.258Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJBV06JM8T8PV46TGCTTQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:54.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:24.257" grpc.time_ms=0.902 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2119gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.19 seconds)2120gitlab # [ 426.405070] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1502121gitlab # [ 427.157698] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d0be4839039c832430a0f9d2","created_at":"2026-10-02T14:16:25.013Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72f9b1fe8b8a51c985876ede5371305c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.061Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d0be4839039c832430a0f9d2: start","job_status":"start","queue_duration_s":0.002292,"scheduling_latency_s":0.00233}2122gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2123gitlab # [ 427.403622] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"d0be4839039c832430a0f9d2","created_at":"2026-10-02T14:16:25.013Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72f9b1fe8b8a51c985876ede5371305c","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.061Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-d0be4839039c832430a0f9d2: done: 0.24632 sec","job_status":"done","queue_duration_s":0.002292,"scheduling_latency_s":0.00233,"redis_calls":11,"redis_duration_s":0.014752,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007534,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.0049,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.073,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.073,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0013724,"exclusive_lock_hold_duration_s":0.10327034,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.24632,"completed_at":"2026-10-02T14:16:25.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.090678,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 427.474121] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.312Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"aa533fe61640d2e7d2d0b8ba","created_at":"2026-10-02T14:16:25.082Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ff2dfe5173ba6fb35ed21eda3ddbc019","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.089Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-aa533fe61640d2e7d2d0b8ba: start","job_status":"start","queue_duration_s":0.222732,"scheduling_latency_s":0.222762}2125gitlab # [ 427.491371] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.378Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"aa533fe61640d2e7d2d0b8ba","created_at":"2026-10-02T14:16:25.082Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ff2dfe5173ba6fb35ed21eda3ddbc019","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.089Z","job_size_bytes":3,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-aa533fe61640d2e7d2d0b8ba: done: 0.065961 sec","job_status":"done","queue_duration_s":0.222732,"scheduling_latency_s":0.222762,"redis_calls":10,"redis_duration_s":0.012275999999999999,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000696,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032566,"duration_s":0.065961,"completed_at":"2026-10-02T14:16:25.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2126gitlab # [ 427.540825] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3f189a01987c74c07ab2f63e","created_at":"2026-10-02T14:16:25.095Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00d32102eb715dcb2fdb65c85f16d1ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.115Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3f189a01987c74c07ab2f63e: start","job_status":"start","queue_duration_s":0.264546,"scheduling_latency_s":0.264572}2127gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.15 seconds)2128gitlab # [ 427.556031] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.403Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2129gitlab # [ 427.559396] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.423Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3f189a01987c74c07ab2f63e","created_at":"2026-10-02T14:16:25.095Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00d32102eb715dcb2fdb65c85f16d1ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.115Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3f189a01987c74c07ab2f63e: done: 0.043321 sec","job_status":"done","queue_duration_s":0.264546,"scheduling_latency_s":0.264572,"redis_calls":9,"redis_duration_s":0.00688,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000753,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00045,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043321,"completed_at":"2026-10-02T14:16:25.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2130gitlab # [ 427.604326] gitaly[924]: time="2026-10-02T14:16:25.420Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJD09J5EA35V1X92AG9NG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:55.119" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:25.419" grpc.time_ms=0.679 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2131gitlab # [ 427.617791] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:25 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1022132gitlab # [ 427.620608] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"28d55b7d2f19803b7de75abf","created_at":"2026-10-02T14:16:25.126Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.134Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-28d55b7d2f19803b7de75abf: start","job_status":"start","queue_duration_s":0.315841,"scheduling_latency_s":0.315868}2133gitlab # [ 427.657497] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"28d55b7d2f19803b7de75abf","created_at":"2026-10-02T14:16:25.126Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.134Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-28d55b7d2f19803b7de75abf: done: 0.113988 sec","job_status":"done","queue_duration_s":0.315841,"scheduling_latency_s":0.315868,"redis_calls":18,"redis_duration_s":0.028862,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.004591,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.001252,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.019629,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.113988,"completed_at":"2026-10-02T14:16:25.564Z","load_balancing_strategy":"primary","db_duration_s":0.015267,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134gitlab # [ 427.703931] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"49f379620c5a0852c3663607","created_at":"2026-10-02T14:16:25.140Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.152Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-49f379620c5a0852c3663607: start","job_status":"start","queue_duration_s":0.416267,"scheduling_latency_s":0.416291}2135gitlab # [ 427.999216] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.906Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleDeleteObjectsCronWorker","jid":"49f379620c5a0852c3663607","created_at":"2026-10-02T14:16:25.140Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8251d4193fad694e50c4958d17fdca56b6fb205fa6961a4b1ccb7cd497bc09c1","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.152Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleDeleteObjectsCronWorker JID-49f379620c5a0852c3663607: done: 0.336775 sec","job_status":"done","queue_duration_s":0.416267,"scheduling_latency_s":0.416291,"redis_calls":65,"redis_duration_s":0.10335499999999999,"redis_read_bytes":114,"redis_write_bytes":41332,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003982,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":656,"redis_queues_calls":2,"redis_queues_duration_s":0.000713,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37035,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.095673,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3374,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002987,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":267,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.153344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035606,"duration_s":0.336775,"completed_at":"2026-10-02T14:16:25.905Z","load_balancing_strategy":"primary","db_duration_s":0.034177,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136gitlab # [ 428.068451] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.907Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"da5ed7aca217da5c18fc8750","created_at":"2026-10-02T14:16:25.158Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed218560e531e13f41e59acf2164076e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.168Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-da5ed7aca217da5c18fc8750: start","job_status":"start","queue_duration_s":0.739421,"scheduling_latency_s":0.739445}2137gitlab # [ 428.087866] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.938Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"da5ed7aca217da5c18fc8750","created_at":"2026-10-02T14:16:25.158Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed218560e531e13f41e59acf2164076e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.168Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-da5ed7aca217da5c18fc8750: done: 0.030786 sec","job_status":"done","queue_duration_s":0.739421,"scheduling_latency_s":0.739445,"redis_calls":11,"redis_duration_s":0.002495,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000396,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00064482,"exclusive_lock_hold_duration_s":0.00667544,"duration_s":0.030786,"completed_at":"2026-10-02T14:16:25.938Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138gitlab # [ 428.135659] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"72d4f7c2573ab648d715e8d8","created_at":"2026-10-02T14:16:25.195Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e49626f535937ad31e705accd18d1bf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.204Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-72d4f7c2573ab648d715e8d8: start","job_status":"start","queue_duration_s":0.736258,"scheduling_latency_s":0.736281}2139gitlab # [ 428.151094] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.992Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"72d4f7c2573ab648d715e8d8","created_at":"2026-10-02T14:16:25.195Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7e49626f535937ad31e705accd18d1bf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.204Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-72d4f7c2573ab648d715e8d8: done: 0.051729 sec","job_status":"done","queue_duration_s":0.736258,"scheduling_latency_s":0.736281,"redis_calls":13,"redis_duration_s":0.002933,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000781,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001289,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00054812,"exclusive_lock_hold_duration_s":0.02631324,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.051729,"completed_at":"2026-10-02T14:16:25.992Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015357,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab # [ 428.198234] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:25.993Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d1604833ddcb0a7e51a8a256","created_at":"2026-10-02T14:16:25.213Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05823cbb0478802a013cfb3a17f641e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.221Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d1604833ddcb0a7e51a8a256: start","job_status":"start","queue_duration_s":0.771855,"scheduling_latency_s":0.771877}2141gitlab # [ 428.213678] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.006Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2142gitlab # [ 428.217226] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d1604833ddcb0a7e51a8a256","created_at":"2026-10-02T14:16:25.213Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"05823cbb0478802a013cfb3a17f641e5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.221Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d1604833ddcb0a7e51a8a256: done: 0.020224 sec","job_status":"done","queue_duration_s":0.771855,"scheduling_latency_s":0.771877,"redis_calls":9,"redis_duration_s":0.001689,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000376,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020224,"completed_at":"2026-10-02T14:16:26.013Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000502,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 428.261361] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.015Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c893b143e5544af4f91fef08","created_at":"2026-10-02T14:16:25.266Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27cf4a8eb1c7ebfe58f67b01296a0b84","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.278Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c893b143e5544af4f91fef08: start","job_status":"start","queue_duration_s":0.736782,"scheduling_latency_s":0.736802}2144gitlab # [ 428.276988] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.027Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2145gitlab # [ 428.281014] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.034Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c893b143e5544af4f91fef08","created_at":"2026-10-02T14:16:25.266Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"27cf4a8eb1c7ebfe58f67b01296a0b84","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.278Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c893b143e5544af4f91fef08: done: 0.019261 sec","job_status":"done","queue_duration_s":0.736782,"scheduling_latency_s":0.736802,"redis_calls":9,"redis_duration_s":0.001675,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00039,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011175,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019261,"completed_at":"2026-10-02T14:16:26.034Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000537,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2146gitlab # [ 428.327053] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8e3224ba247f8a1d647e584b","created_at":"2026-10-02T14:16:25.301Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16bc952c6d785ba5d26f4c5482d32079","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.328Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8e3224ba247f8a1d647e584b: start","job_status":"start","queue_duration_s":0.707283,"scheduling_latency_s":0.7073}2147gitlab # [ 428.341527] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.163Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8e3224ba247f8a1d647e584b","created_at":"2026-10-02T14:16:25.301Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"16bc952c6d785ba5d26f4c5482d32079","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.328Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8e3224ba247f8a1d647e584b: done: 0.127783 sec","job_status":"done","queue_duration_s":0.707283,"scheduling_latency_s":0.7073,"redis_calls":70,"redis_duration_s":0.01464,"redis_read_bytes":119,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001969,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000708,"redis_queues_read_bytes":4,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.011427,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.11366,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.127783,"completed_at":"2026-10-02T14:16:26.163Z","load_balancing_strategy":"primary","db_duration_s":0.001793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 428.386033] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"38d0e6616d2a31605c7deb3f","created_at":"2026-10-02T14:16:25.333Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12ce43f1540d3f2bad0afbcea616f5f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.353Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-38d0e6616d2a31605c7deb3f: start","job_status":"start","queue_duration_s":0.812081,"scheduling_latency_s":0.812105}2149gitlab # [ 428.400357] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.194Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"38d0e6616d2a31605c7deb3f","created_at":"2026-10-02T14:16:25.333Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12ce43f1540d3f2bad0afbcea616f5f0","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.353Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-38d0e6616d2a31605c7deb3f: done: 0.029179 sec","job_status":"done","queue_duration_s":0.812081,"scheduling_latency_s":0.812105,"redis_calls":11,"redis_duration_s":0.002151,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000345,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000848,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057118,"exclusive_lock_hold_duration_s":0.00803878,"duration_s":0.029179,"completed_at":"2026-10-02T14:16:26.194Z","load_balancing_strategy":"primary","db_duration_s":0.003842,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2150gitlab # [ 428.443514] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.196Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4bcc2614922b24a3a9e26d98","created_at":"2026-10-02T14:16:25.362Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76593009b1fff7b4b24601743de07727","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.382Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4bcc2614922b24a3a9e26d98: start","job_status":"start","queue_duration_s":0.813959,"scheduling_latency_s":0.813979}2151gitlab # [ 428.458174] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.230Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4bcc2614922b24a3a9e26d98","created_at":"2026-10-02T14:16:25.362Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"76593009b1fff7b4b24601743de07727","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.382Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4bcc2614922b24a3a9e26d98: done: 0.034576 sec","job_status":"done","queue_duration_s":0.813959,"scheduling_latency_s":0.813979,"redis_calls":9,"redis_duration_s":0.001729,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000349,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001011,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000369,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016785,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.034576,"completed_at":"2026-10-02T14:16:26.230Z","load_balancing_strategy":"primary","db_duration_s":0.011029,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2152gitlab # [ 428.502361] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.232Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c16b6612bf7f5b5dc5813900","created_at":"2026-10-02T14:16:25.388Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f7e26898c78ddea5639b4fa830c097e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.404Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c16b6612bf7f5b5dc5813900: start","job_status":"start","queue_duration_s":0.828143,"scheduling_latency_s":0.828164}2153gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2154gitlab # [ 428.517657] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.255Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c16b6612bf7f5b5dc5813900","created_at":"2026-10-02T14:16:25.388Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f7e26898c78ddea5639b4fa830c097e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.404Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c16b6612bf7f5b5dc5813900: done: 0.023229 sec","job_status":"done","queue_duration_s":0.828143,"scheduling_latency_s":0.828164,"redis_calls":11,"redis_duration_s":0.002383,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000706,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014983,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023229,"completed_at":"2026-10-02T14:16:26.255Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2155gitlab # [ 428.562387] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"47ff4e36d18b02211bc8ef4c","created_at":"2026-10-02T14:16:25.416Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dba0fdc9f4705a02faea053b25baa14c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:16:25.426Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-47ff4e36d18b02211bc8ef4c: start","job_status":"start","queue_duration_s":0.830428,"scheduling_latency_s":0.830446}2156gitlab # [ 428.575657] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.268Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2157gitlab # [ 428.579754] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.275Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"47ff4e36d18b02211bc8ef4c","created_at":"2026-10-02T14:16:25.416Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dba0fdc9f4705a02faea053b25baa14c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T14:16:25.426Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-47ff4e36d18b02211bc8ef4c: done: 0.018991 sec","job_status":"done","queue_duration_s":0.830428,"scheduling_latency_s":0.830446,"redis_calls":9,"redis_duration_s":0.001707,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000341,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000455,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018991,"completed_at":"2026-10-02T14:16:26.275Z","load_balancing_strategy":"primary","db_duration_s":0.000337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158gitlab # [ 428.623175] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.277Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"49448efec77f2cab08ce1714","created_at":"2026-10-02T14:16:25.451Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a382e263ddcd1ed516dd08cc2b0f012","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.467Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-49448efec77f2cab08ce1714: start","job_status":"start","queue_duration_s":0.809029,"scheduling_latency_s":0.809047}2159gitlab # [ 428.638140] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.301Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"49448efec77f2cab08ce1714","created_at":"2026-10-02T14:16:25.451Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5a382e263ddcd1ed516dd08cc2b0f012","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.467Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-49448efec77f2cab08ce1714: done: 0.024795 sec","job_status":"done","queue_duration_s":0.809029,"scheduling_latency_s":0.809047,"redis_calls":9,"redis_duration_s":0.001693,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000369,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024795,"completed_at":"2026-10-02T14:16:26.301Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 428.682151] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.303Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"884dfd26ed27a3cfda13dcd0","created_at":"2026-10-02T14:16:25.486Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bccc736d89f6849a906dbbb1e3250ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.502Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-884dfd26ed27a3cfda13dcd0: start","job_status":"start","queue_duration_s":0.800409,"scheduling_latency_s":0.800427}2161gitlab # [ 428.697286] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.313Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2162gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.15 seconds)2163gitlab # [ 428.701020] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"884dfd26ed27a3cfda13dcd0","created_at":"2026-10-02T14:16:25.486Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0bccc736d89f6849a906dbbb1e3250ac","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.502Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-884dfd26ed27a3cfda13dcd0: done: 0.017423 sec","job_status":"done","queue_duration_s":0.800409,"scheduling_latency_s":0.800427,"redis_calls":9,"redis_duration_s":0.001473,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000317,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000828,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000328,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017423,"completed_at":"2026-10-02T14:16:26.320Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 428.746299] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.321Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2096e1657e6c19cc791876bf","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2096e1657e6c19cc791876bf: start","job_status":"start","queue_duration_s":0.785342,"scheduling_latency_s":0.785359}2165gitlab # [ 428.759416] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.337Z","message":"No existing merge request to be cleaned up."}2166gitlab # [ 428.761713] gitaly[924]: time="2026-10-02T14:16:26.578Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJE4D60JAX2BVT75BRSM7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:56.277" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:26.577" grpc.time_ms=0.681 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2167gitlab # [ 428.775531] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 992168gitlab # [ 428.778473] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.354Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2096e1657e6c19cc791876bf","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2096e1657e6c19cc791876bf: done: 0.032684 sec","job_status":"done","queue_duration_s":0.785342,"scheduling_latency_s":0.785359,"redis_calls":12,"redis_duration_s":0.002484,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021207,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032684,"completed_at":"2026-10-02T14:16:26.354Z","load_balancing_strategy":"primary","db_duration_s":0.004691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2169gitlab # [ 428.821985] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.356Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9555b5ddfa9e98d81277ec4","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9555b5ddfa9e98d81277ec4: start","job_status":"start","queue_duration_s":0.819716,"scheduling_latency_s":0.819739}2170gitlab # [ 428.842756] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.371Z","message":"No existing merge request to be cleaned up."}2171gitlab # [ 428.845718] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.382Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9555b5ddfa9e98d81277ec4","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9555b5ddfa9e98d81277ec4: done: 0.026195 sec","job_status":"done","queue_duration_s":0.819716,"scheduling_latency_s":0.819739,"redis_calls":10,"redis_duration_s":0.002439,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001502,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026195,"completed_at":"2026-10-02T14:16:26.382Z","load_balancing_strategy":"primary","db_duration_s":0.002217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 428.900790] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.383Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7cc9adbde14a1b1b21949f8e","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7cc9adbde14a1b1b21949f8e: start","job_status":"start","queue_duration_s":0.847546,"scheduling_latency_s":0.847569}2173gitlab # [ 428.914385] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.397Z","message":"No existing merge request to be cleaned up."}2174gitlab # [ 428.916152] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.408Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"7cc9adbde14a1b1b21949f8e","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-7cc9adbde14a1b1b21949f8e: done: 0.024263 sec","job_status":"done","queue_duration_s":0.847546,"scheduling_latency_s":0.847569,"redis_calls":10,"redis_duration_s":0.002036,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024263,"completed_at":"2026-10-02T14:16:26.408Z","load_balancing_strategy":"primary","db_duration_s":0.002032,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2175gitlab # [ 428.960347] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.409Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce56979c2243c69e8df6c28a","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce56979c2243c69e8df6c28a: start","job_status":"start","queue_duration_s":0.873315,"scheduling_latency_s":0.873337}2176gitlab # [ 428.974167] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.423Z","message":"No existing merge request to be cleaned up."}2177gitlab # [ 428.976057] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.432Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce56979c2243c69e8df6c28a","created_at":"2026-10-02T14:16:25.499Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-b5f8ccdde97b4057","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7f3f34e031fdbe491f269f5f8ff2b9b8","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.536Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce56979c2243c69e8df6c28a: done: 0.023072 sec","job_status":"done","queue_duration_s":0.873315,"scheduling_latency_s":0.873337,"redis_calls":10,"redis_duration_s":0.001989,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000833,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001156,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023072,"completed_at":"2026-10-02T14:16:26.432Z","load_balancing_strategy":"primary","db_duration_s":0.001879,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2178gitlab # [ 429.019449] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c452170e60c2fc83186d8c81","created_at":"2026-10-02T14:16:25.527Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"839d74a7a1e5981f4e3a0acde4c09ab2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.540Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c452170e60c2fc83186d8c81: start","job_status":"start","queue_duration_s":0.893474,"scheduling_latency_s":0.893495}2179gitlab # [ 429.035519] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.460Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c452170e60c2fc83186d8c81","created_at":"2026-10-02T14:16:25.527Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"839d74a7a1e5981f4e3a0acde4c09ab2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.540Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c452170e60c2fc83186d8c81: done: 0.026341 sec","job_status":"done","queue_duration_s":0.893474,"scheduling_latency_s":0.893495,"redis_calls":9,"redis_duration_s":0.001701,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00036,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016395,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026341,"completed_at":"2026-10-02T14:16:26.460Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180gitlab # [ 429.082945] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a91fb7ece782735c323b67bc","created_at":"2026-10-02T14:16:25.557Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51b8fd68a9c28f98b9be0972278f8735","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:16:25.567Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a91fb7ece782735c323b67bc: start","job_status":"start","queue_duration_s":0.991353,"scheduling_latency_s":0.991381}2181gitlab # [ 429.098220] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.577Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2182gitlab # [ 429.102030] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"a91fb7ece782735c323b67bc","created_at":"2026-10-02T14:16:25.557Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"51b8fd68a9c28f98b9be0972278f8735","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T14:16:25.567Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-a91fb7ece782735c323b67bc: done: 0.029001 sec","job_status":"done","queue_duration_s":0.991353,"scheduling_latency_s":0.991381,"redis_calls":9,"redis_duration_s":0.003074,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000775,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000552,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029001,"completed_at":"2026-10-02T14:16:26.588Z","load_balancing_strategy":"primary","db_duration_s":0.000741,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183gitlab # [ 429.149622] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"ec0b84ba1319a5432505a319","created_at":"2026-10-02T14:16:25.575Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6c9ff74a1a3b8786e50c575ef28271f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.582Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-ec0b84ba1319a5432505a319: start","job_status":"start","queue_duration_s":1.008275,"scheduling_latency_s":1.008308}2184gitlab # [ 429.164904] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.633Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"ec0b84ba1319a5432505a319","created_at":"2026-10-02T14:16:25.575Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6c9ff74a1a3b8786e50c575ef28271f","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.582Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-ec0b84ba1319a5432505a319: done: 0.042929 sec","job_status":"done","queue_duration_s":1.008275,"scheduling_latency_s":1.008308,"redis_calls":9,"redis_duration_s":0.002638,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001461,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000695,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042929,"completed_at":"2026-10-02T14:16:26.633Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2185gitlab # [ 429.211237] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"aa523443332a0b72d787d3aa","created_at":"2026-10-02T14:16:25.590Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f1c8820b06449804f11b5424309eedc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.605Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-aa523443332a0b72d787d3aa: start","job_status":"start","queue_duration_s":1.029625,"scheduling_latency_s":1.02965}2186gitlab # [ 429.226632] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"aa523443332a0b72d787d3aa","created_at":"2026-10-02T14:16:25.590Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3f1c8820b06449804f11b5424309eedc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.605Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-aa523443332a0b72d787d3aa: done: 0.273557 sec","job_status":"done","queue_duration_s":1.029625,"scheduling_latency_s":1.02965,"redis_calls":13,"redis_duration_s":0.0033819999999999996,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001102,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001168,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001112,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.224633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00068168,"exclusive_lock_hold_duration_s":0.24245566,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.273557,"completed_at":"2026-10-02T14:16:26.908Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.054481,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187gitlab # [ 429.274168] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.910Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"aba42cb12e21fbdafeff5295","created_at":"2026-10-02T14:16:25.642Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56466ff2d75028062b397be5254d37fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:16:25.656Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-aba42cb12e21fbdafeff5295: start","job_status":"start","queue_duration_s":1.254057,"scheduling_latency_s":1.25408}2188gitlab # [ 429.288208] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"aba42cb12e21fbdafeff5295","created_at":"2026-10-02T14:16:25.642Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"56466ff2d75028062b397be5254d37fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T14:16:25.656Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-aba42cb12e21fbdafeff5295: done: 0.03499 sec","job_status":"done","queue_duration_s":1.254057,"scheduling_latency_s":1.25408,"redis_calls":9,"redis_duration_s":0.00181,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000393,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001038,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03499,"completed_at":"2026-10-02T14:16:26.945Z","load_balancing_strategy":"primary","db_duration_s":0.004601,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2189gitlab # [ 429.330882] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.946Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d2a833131abb12622e7cdb29","created_at":"2026-10-02T14:16:25.674Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fd586981cd537cc22cd5a35f37d1803","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.685Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d2a833131abb12622e7cdb29: start","job_status":"start","queue_duration_s":1.261005,"scheduling_latency_s":1.261029}2190gitlab # [ 429.345927] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.969Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d2a833131abb12622e7cdb29","created_at":"2026-10-02T14:16:25.674Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4fd586981cd537cc22cd5a35f37d1803","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.685Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d2a833131abb12622e7cdb29: done: 0.022575 sec","job_status":"done","queue_duration_s":1.261005,"scheduling_latency_s":1.261029,"redis_calls":11,"redis_duration_s":0.00254,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000388,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000777,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022575,"completed_at":"2026-10-02T14:16:26.969Z","load_balancing_strategy":"primary","db_duration_s":0.000583,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191gitlab # [ 429.390988] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"79b6c5f051c7bed45c75a1f6","created_at":"2026-10-02T14:16:25.697Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73e6025eb49e85b7ccfb8fa02b07e89d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.712Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-79b6c5f051c7bed45c75a1f6: start","job_status":"start","queue_duration_s":1.259656,"scheduling_latency_s":1.259682}2192gitlab # [ 429.406411] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.998Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"79b6c5f051c7bed45c75a1f6","created_at":"2026-10-02T14:16:25.697Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"73e6025eb49e85b7ccfb8fa02b07e89d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.712Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-79b6c5f051c7bed45c75a1f6: done: 0.02601 sec","job_status":"done","queue_duration_s":1.259656,"scheduling_latency_s":1.259682,"redis_calls":11,"redis_duration_s":0.002254,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000371,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005583,"exclusive_lock_hold_duration_s":0.00460722,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.02601,"completed_at":"2026-10-02T14:16:26.997Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193gitlab # [ 429.452343] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:26.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"04aae242fdd767ff1f7feae4","created_at":"2026-10-02T14:16:25.747Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54b50a9f7856878a64d8d6643bac5099","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.770Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-04aae242fdd767ff1f7feae4: start","job_status":"start","queue_duration_s":1.229148,"scheduling_latency_s":1.229172}2194gitlab # [ 429.467467] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.360Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"04aae242fdd767ff1f7feae4","created_at":"2026-10-02T14:16:25.747Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"54b50a9f7856878a64d8d6643bac5099","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.770Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-04aae242fdd767ff1f7feae4: done: 0.361264 sec","job_status":"done","queue_duration_s":1.229148,"scheduling_latency_s":1.229172,"redis_calls":564,"redis_duration_s":0.078702,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.077192,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.323907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.361264,"completed_at":"2026-10-02T14:16:27.360Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 429.512249] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.362Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ad7cdab3317ae617718b73c5","created_at":"2026-10-02T14:16:25.798Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb499bb22c8089bdbde8548142f23f42","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.814Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ad7cdab3317ae617718b73c5: start","job_status":"start","queue_duration_s":1.54768,"scheduling_latency_s":1.547709}2196gitlab # [ 429.527499] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ad7cdab3317ae617718b73c5","created_at":"2026-10-02T14:16:25.798Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb499bb22c8089bdbde8548142f23f42","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.814Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ad7cdab3317ae617718b73c5: done: 0.038319 sec","job_status":"done","queue_duration_s":1.54768,"scheduling_latency_s":1.547709,"redis_calls":12,"redis_duration_s":0.002799,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000457,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001318,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001024,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01146516,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.038319,"completed_at":"2026-10-02T14:16:27.400Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004043,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2197gitlab # [ 429.576307] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6ee2ab22de5bdd3247ea57f6","created_at":"2026-10-02T14:16:25.826Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d752055260b55fa825520ec125038797","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.839Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6ee2ab22de5bdd3247ea57f6: start","job_status":"start","queue_duration_s":1.562555,"scheduling_latency_s":1.562583}2198gitlab # [ 429.592374] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.425Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6ee2ab22de5bdd3247ea57f6","created_at":"2026-10-02T14:16:25.826Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-71791d5a353e4157","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d752055260b55fa825520ec125038797","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.839Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6ee2ab22de5bdd3247ea57f6: done: 0.022852 sec","job_status":"done","queue_duration_s":1.562555,"scheduling_latency_s":1.562583,"redis_calls":9,"redis_duration_s":0.00198,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000416,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022852,"completed_at":"2026-10-02T14:16:27.425Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000532,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2199gitlab # [ 429.639099] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.426Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4a69245cd8a1a54749224746","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4a69245cd8a1a54749224746: start","job_status":"start","queue_duration_s":1.532068,"scheduling_latency_s":1.532097}2200gitlab # [ 429.653078] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4a69245cd8a1a54749224746","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4a69245cd8a1a54749224746: done: 0.042922 sec","job_status":"done","queue_duration_s":1.532068,"scheduling_latency_s":1.532097,"redis_calls":13,"redis_duration_s":0.003537,"redis_read_bytes":12,"redis_write_bytes":1300,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000432,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":289,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00144,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":395,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001665,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":616,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.011,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066474,"duration_s":0.042922,"completed_at":"2026-10-02T14:16:27.469Z","load_balancing_strategy":"primary","db_duration_s":0.007074,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2201gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2202gitlab # [ 429.698331] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fb9351685794894db40a59d1","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fb9351685794894db40a59d1: start","job_status":"start","queue_duration_s":1.577123,"scheduling_latency_s":1.577153}2203gitlab # [ 429.711955] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fb9351685794894db40a59d1","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fb9351685794894db40a59d1: done: 0.025196 sec","job_status":"done","queue_duration_s":1.577123,"scheduling_latency_s":1.577153,"redis_calls":10,"redis_duration_s":0.002553,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001172,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001381,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025196,"completed_at":"2026-10-02T14:16:27.497Z","load_balancing_strategy":"primary","db_duration_s":0.002682,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2204gitlab # [ 429.754604] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"00c8c9727a299a66618d6afd","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-00c8c9727a299a66618d6afd: start","job_status":"start","queue_duration_s":1.604356,"scheduling_latency_s":1.604386}2205gitlab # [ 429.769500] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"00c8c9727a299a66618d6afd","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-00c8c9727a299a66618d6afd: done: 0.024806 sec","job_status":"done","queue_duration_s":1.604356,"scheduling_latency_s":1.604386,"redis_calls":10,"redis_duration_s":0.002301,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000931,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00137,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024806,"completed_at":"2026-10-02T14:16:27.523Z","load_balancing_strategy":"primary","db_duration_s":0.002342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206gitlab # [ 429.813589] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f4b24a713eb452243f64d05","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f4b24a713eb452243f64d05: start","job_status":"start","queue_duration_s":1.630761,"scheduling_latency_s":1.63079}2207gitlab # [ 429.827586] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5f4b24a713eb452243f64d05","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5f4b24a713eb452243f64d05: done: 0.025371 sec","job_status":"done","queue_duration_s":1.630761,"scheduling_latency_s":1.63079,"redis_calls":10,"redis_duration_s":0.0022689999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001334,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025371,"completed_at":"2026-10-02T14:16:27.550Z","load_balancing_strategy":"primary","db_duration_s":0.002228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208gitlab # [ 429.869415] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2dd6f1af05a4ad63531d76b1","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2dd6f1af05a4ad63531d76b1: start","job_status":"start","queue_duration_s":1.657839,"scheduling_latency_s":1.657868}2209gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.19 seconds)2210gitlab # [ 429.882106] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"2dd6f1af05a4ad63531d76b1","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-2dd6f1af05a4ad63531d76b1: done: 0.029691 sec","job_status":"done","queue_duration_s":1.657839,"scheduling_latency_s":1.657868,"redis_calls":10,"redis_duration_s":0.0030689999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00156,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029691,"completed_at":"2026-10-02T14:16:27.582Z","load_balancing_strategy":"primary","db_duration_s":0.002722,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2211gitlab # [ 429.922601] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"848573be3d967a1f5a6b940a","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-848573be3d967a1f5a6b940a: start","job_status":"start","queue_duration_s":1.69007,"scheduling_latency_s":1.69011}2212gitlab # [ 429.936107] gitaly[924]: time="2026-10-02T14:16:27.751Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJF8J37C9N09R6M7161HP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:57.549" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:27.750" grpc.time_ms=0.954 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2213gitlab # [ 429.949947] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:27 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1362214gitlab # [ 429.952899] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"848573be3d967a1f5a6b940a","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-848573be3d967a1f5a6b940a: done: 0.032834 sec","job_status":"done","queue_duration_s":1.69007,"scheduling_latency_s":1.69011,"redis_calls":10,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0011,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001777,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032834,"completed_at":"2026-10-02T14:16:27.617Z","load_balancing_strategy":"primary","db_duration_s":0.002803,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2215gitlab # [ 429.993698] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.619Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"64ed59c9ad5566acf70c6343","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-64ed59c9ad5566acf70c6343: start","job_status":"start","queue_duration_s":1.725059,"scheduling_latency_s":1.725094}2216gitlab # [ 430.007325] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"64ed59c9ad5566acf70c6343","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-64ed59c9ad5566acf70c6343: done: 0.040964 sec","job_status":"done","queue_duration_s":1.725059,"scheduling_latency_s":1.725094,"redis_calls":10,"redis_duration_s":0.006835000000000001,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002637,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.004198,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025288,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040964,"completed_at":"2026-10-02T14:16:27.660Z","load_balancing_strategy":"primary","db_duration_s":0.006283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2217gitlab # [ 430.050405] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"44a2f03d0ca9429b18743e8e","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-44a2f03d0ca9429b18743e8e: start","job_status":"start","queue_duration_s":1.76849,"scheduling_latency_s":1.768523}2218gitlab # [ 430.064067] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"44a2f03d0ca9429b18743e8e","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-44a2f03d0ca9429b18743e8e: done: 0.032694 sec","job_status":"done","queue_duration_s":1.76849,"scheduling_latency_s":1.768523,"redis_calls":10,"redis_duration_s":0.004889,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.003732,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032694,"completed_at":"2026-10-02T14:16:27.695Z","load_balancing_strategy":"primary","db_duration_s":0.002817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2219gitlab # [ 430.106577] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.698Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a219f6465d4bb98430716d27","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a219f6465d4bb98430716d27: start","job_status":"start","queue_duration_s":1.803317,"scheduling_latency_s":1.803348}2220gitlab # [ 430.120442] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a219f6465d4bb98430716d27","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a219f6465d4bb98430716d27: done: 0.029897 sec","job_status":"done","queue_duration_s":1.803317,"scheduling_latency_s":1.803348,"redis_calls":10,"redis_duration_s":0.002506,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001459,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029897,"completed_at":"2026-10-02T14:16:27.727Z","load_balancing_strategy":"primary","db_duration_s":0.002838,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2221gitlab # [ 430.163706] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b6d6e7641dd7e7093c859e87","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b6d6e7641dd7e7093c859e87: start","job_status":"start","queue_duration_s":1.835412,"scheduling_latency_s":1.835447}2222gitlab # [ 430.178317] nix-daemon[3710]: accepted connection from pid 4774, user nixbot2223gitlab # [ 430.179462] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"b6d6e7641dd7e7093c859e87","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-b6d6e7641dd7e7093c859e87: done: 0.0357 sec","job_status":"done","queue_duration_s":1.835412,"scheduling_latency_s":1.835447,"redis_calls":10,"redis_duration_s":0.004581,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001873,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002708,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.003,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0357,"completed_at":"2026-10-02T14:16:27.765Z","load_balancing_strategy":"primary","db_duration_s":0.003606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 430.224806] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4d06cca51d04664e2db21185","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4d06cca51d04664e2db21185: start","job_status":"start","queue_duration_s":1.873481,"scheduling_latency_s":1.873518}2225gitlab # [ 430.238765] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"4d06cca51d04664e2db21185","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-4d06cca51d04664e2db21185: done: 0.032378 sec","job_status":"done","queue_duration_s":1.873481,"scheduling_latency_s":1.873518,"redis_calls":10,"redis_duration_s":0.002613,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00152,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032378,"completed_at":"2026-10-02T14:16:27.800Z","load_balancing_strategy":"primary","db_duration_s":0.002858,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2226gitlab # [ 430.282563] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"602533b5803adc52fe196a62","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-602533b5803adc52fe196a62: start","job_status":"start","queue_duration_s":1.907855,"scheduling_latency_s":1.907901}2227gitlab # [ 430.296673] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"602533b5803adc52fe196a62","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-602533b5803adc52fe196a62: done: 0.033227 sec","job_status":"done","queue_duration_s":1.907855,"scheduling_latency_s":1.907901,"redis_calls":10,"redis_duration_s":0.002966,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001745,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033227,"completed_at":"2026-10-02T14:16:27.835Z","load_balancing_strategy":"primary","db_duration_s":0.002978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2228gitlab # [ 430.340971] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"be3109d9c7b07bef3a66c6fa","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-be3109d9c7b07bef3a66c6fa: start","job_status":"start","queue_duration_s":1.942878,"scheduling_latency_s":1.942911}2229gitlab # [ 430.355003] gitaly[924]: time="2026-10-02T14:16:28.171Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJFN9TK4JC3X9ABWM45TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:58.169" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:28.170" grpc.time_ms=1.238 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2230gitlab # [ 430.369765] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"be3109d9c7b07bef3a66c6fa","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-be3109d9c7b07bef3a66c6fa: done: 0.029531 sec","job_status":"done","queue_duration_s":1.942878,"scheduling_latency_s":1.942911,"redis_calls":10,"redis_duration_s":0.0027229999999999997,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000995,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029531,"completed_at":"2026-10-02T14:16:27.867Z","load_balancing_strategy":"primary","db_duration_s":0.00283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab # [ 430.414643] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"82906763581f231e3747c61b","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-82906763581f231e3747c61b: start","job_status":"start","queue_duration_s":1.973823,"scheduling_latency_s":1.973846}2232gitlab # [ 430.428713] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"82906763581f231e3747c61b","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-82906763581f231e3747c61b: done: 0.020262 sec","job_status":"done","queue_duration_s":1.973823,"scheduling_latency_s":1.973846,"redis_calls":10,"redis_duration_s":0.001778,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020262,"completed_at":"2026-10-02T14:16:27.888Z","load_balancing_strategy":"primary","db_duration_s":0.001932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233gitlab # [ 430.472732] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39fe1bb567ec661806f364f5","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39fe1bb567ec661806f364f5: start","job_status":"start","queue_duration_s":1.995488,"scheduling_latency_s":1.995511}2234gitlab # [ 430.486685] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.911Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"39fe1bb567ec661806f364f5","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-39fe1bb567ec661806f364f5: done: 0.0212 sec","job_status":"done","queue_duration_s":1.995488,"scheduling_latency_s":1.995511,"redis_calls":10,"redis_duration_s":0.00219,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001435,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0212,"completed_at":"2026-10-02T14:16:27.911Z","load_balancing_strategy":"primary","db_duration_s":0.00197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2235gitlab # [ 430.530585] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.912Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ab7cbc8f506c1ccedfdd67e7","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ab7cbc8f506c1ccedfdd67e7: start","job_status":"start","queue_duration_s":2.018117,"scheduling_latency_s":2.018142}2236gitlab # [ 430.544615] gitaly[924]: time="2026-10-02T14:16:28.368Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJFN9TK4JC3X9ABWM45TN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:38.045" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:28.355" grpc.time_ms=13.288 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2237gitlab # [ 430.559933] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"ab7cbc8f506c1ccedfdd67e7","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-ab7cbc8f506c1ccedfdd67e7: done: 0.021125 sec","job_status":"done","queue_duration_s":2.018117,"scheduling_latency_s":2.018142,"redis_calls":10,"redis_duration_s":0.001824,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001069,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.001,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.001,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021125,"completed_at":"2026-10-02T14:16:27.933Z","load_balancing_strategy":"primary","db_duration_s":0.002007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2238gitlab # [ 430.602757] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"60d9351be77cda24c1c551f3","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-60d9351be77cda24c1c551f3: start","job_status":"start","queue_duration_s":2.04066,"scheduling_latency_s":2.040684}2239gitlab # [ 430.617280] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 3862240gitlab # [ 430.620418] nixbot[3547]: {"ts": "2026-10-02T14:16:28.458Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df \"HTTP/1.1 201 Created\""}2241gitlab # [ 430.623477] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"60d9351be77cda24c1c551f3","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-60d9351be77cda24c1c551f3: done: 0.020875 sec","job_status":"done","queue_duration_s":2.04066,"scheduling_latency_s":2.040684,"redis_calls":10,"redis_duration_s":0.001839,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000766,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001073,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020875,"completed_at":"2026-10-02T14:16:27.956Z","load_balancing_strategy":"primary","db_duration_s":0.00193,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2242gitlab # [ 430.666245] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e6eca9290b43ecaa118a8ed","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e6eca9290b43ecaa118a8ed: start","job_status":"start","queue_duration_s":2.063235,"scheduling_latency_s":2.063259}2243gitlab # [ 430.679059] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"6e6eca9290b43ecaa118a8ed","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-6e6eca9290b43ecaa118a8ed: done: 0.022493 sec","job_status":"done","queue_duration_s":2.063235,"scheduling_latency_s":2.063259,"redis_calls":10,"redis_duration_s":0.002085,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022493,"completed_at":"2026-10-02T14:16:27.980Z","load_balancing_strategy":"primary","db_duration_s":0.002146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2244gitlab # [ 430.719841] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:27.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a816a76da66f767cba813c50","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a816a76da66f767cba813c50: start","job_status":"start","queue_duration_s":2.087948,"scheduling_latency_s":2.087975}2245gitlab # [ 430.733138] gitaly[924]: time="2026-10-02T14:16:28.535Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJG1KAY4892BXQJ3NBM7C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:58.333" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:28.534" grpc.time_ms=0.787 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2246gitlab # [ 430.746793] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"a816a76da66f767cba813c50","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-a816a76da66f767cba813c50: done: 0.022049 sec","job_status":"done","queue_duration_s":2.087948,"scheduling_latency_s":2.087975,"redis_calls":10,"redis_duration_s":0.001706,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001002,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022049,"completed_at":"2026-10-02T14:16:28.004Z","load_balancing_strategy":"primary","db_duration_s":0.002075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2247gitlab # [ 430.787874] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5256ec086039bed884ea56d1","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5256ec086039bed884ea56d1: start","job_status":"start","queue_duration_s":2.112,"scheduling_latency_s":2.112026}2248gitlab # [ 430.801020] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1812249gitlab # [ 430.803643] gitaly[924]: time="2026-10-02T14:16:28.587Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJG1KAY4892BXQJ3NBM7C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:38.264" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:28.575" grpc.time_ms=11.933 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2250gitlab # [ 430.818105] nixbot[3547]: {"ts": "2026-10-02T14:16:28.647Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df \"HTTP/1.1 201 Created\""}2251gitlab # [ 430.822068] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"5256ec086039bed884ea56d1","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-5256ec086039bed884ea56d1: done: 0.027762 sec","job_status":"done","queue_duration_s":2.112,"scheduling_latency_s":2.112026,"redis_calls":10,"redis_duration_s":0.0026420000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027762,"completed_at":"2026-10-02T14:16:28.034Z","load_balancing_strategy":"primary","db_duration_s":0.002258,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2252gitlab # [ 430.862759] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"610783740ccb4442cdc19417","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-610783740ccb4442cdc19417: start","job_status":"start","queue_duration_s":2.141888,"scheduling_latency_s":2.14192}2253gitlab # [ 430.875742] gitaly[924]: time="2026-10-02T14:16:28.723Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YFJG7MADTRZHK4A0J0HXF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T14:16:58.521" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T14:16:28.722" grpc.time_ms=0.663 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2254gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses2255gitlab # [ 430.889286] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"610783740ccb4442cdc19417","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-610783740ccb4442cdc19417: done: 0.03123 sec","job_status":"done","queue_duration_s":2.141888,"scheduling_latency_s":2.14192,"redis_calls":10,"redis_duration_s":0.002643,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001611,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03123,"completed_at":"2026-10-02T14:16:28.067Z","load_balancing_strategy":"primary","db_duration_s":0.00238,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256gitlab # [ 430.928210] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"592e33b872527e848d40e496","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-592e33b872527e848d40e496: start","job_status":"start","queue_duration_s":2.175055,"scheduling_latency_s":2.175087}2257gitlab # [ 430.941462] gitaly[924]: time="2026-10-02T14:16:28.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YFJG7MADTRZHK4A0J0HXF7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T14:16:38.117" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T14:16:28.768" grpc.time_ms=9.263 pid=924 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2258gitlab # [ 430.956830] nixbot[3547]: {"ts": "2026-10-02T14:16:28.839Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df \"HTTP/1.1 201 Created\""}2259gitlab # [ 430.960625] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/10/02:14:16:28 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/45e21e5de49e68885eab25da3122e70bb390a5df HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1762260gitlab # [ 430.963613] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"592e33b872527e848d40e496","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-592e33b872527e848d40e496: done: 0.037575 sec","job_status":"done","queue_duration_s":2.175055,"scheduling_latency_s":2.175087,"redis_calls":10,"redis_duration_s":0.00304,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00135,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00169,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037575,"completed_at":"2026-10-02T14:16:28.107Z","load_balancing_strategy":"primary","db_duration_s":0.002921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2261gitlab # [ 431.007543] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0970765c5cf361e6d1d05b94","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0970765c5cf361e6d1d05b94: start","job_status":"start","queue_duration_s":2.214955,"scheduling_latency_s":2.214992}2262gitlab # [ 431.020855] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"0970765c5cf361e6d1d05b94","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-0970765c5cf361e6d1d05b94: done: 0.036477 sec","job_status":"done","queue_duration_s":2.214955,"scheduling_latency_s":2.214992,"redis_calls":10,"redis_duration_s":0.0027470000000000003,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001552,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.002,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036477,"completed_at":"2026-10-02T14:16:28.146Z","load_balancing_strategy":"primary","db_duration_s":0.002735,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2263gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/45e21e5de49e68885eab25da3122e70bb390a5df/statuses, in 0.18 seconds)2264gitlab # [ 431.065705] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fdd8a893b6a718ae3a87cd41","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fdd8a893b6a718ae3a87cd41: start","job_status":"start","queue_duration_s":2.254279,"scheduling_latency_s":2.254316}2265(finished: subtest: eval and build post commit statuses, in 9.77 seconds)2266(finished: run the VM test script, in 505.73 seconds)2267test script finished in 505.76s2268cleanup2269kill QemuMachine (pid 46)2270gitlab # [ 431.078816] sidekiq-cluster[3422]: {"severity":"INFO","time":"2026-10-02T14:16:28.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DeleteObjectsWorker","args":[],"jid":"fdd8a893b6a718ae3a87cd41","created_at":"2026-10-02T14:16:25.649Z","trace_propagation_headers":{"sentry-trace":"702a2eeeae5843bea3ae8c87ed59f60c-ee16eb71798c43bf","baggage":"sentry-trace_id=702a2eeeae5843bea3ae8c87ed59f60c,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleDeleteObjectsCronWorker","correlation_id":"72249d7323e400c6fd800e045a5c77b6","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T14:16:25.894Z","job_size_bytes":2,"pid":3422,"sidekiq_tid":"m8mu","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DeleteObjectsWorker JID-fdd8a893b6a718ae3a87cd41: done: 0.042104 sec","job_status":"done","queue_duration_s":2.254279,"scheduling_latency_s":2.254316,"redis_calls":10,"redis_duration_s":0.004099,"redis_read_bytes":8,"redis_write_bytes":942,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001759,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":326,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00234,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":616,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":2,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2271(finished: cleanup, in 1.42 seconds)2272additionally exposed symbols:2273 gitlab,2274 vlan1,2275 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2276[]2277[]2278[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-02T14:14:49.326Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2279[]2280[]2281[]2282[]2283[]2284[]2285[]2286[]2287[]2288[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-02T14:16:18.541226Z', 'started_at': '2026-10-02T14:16:18.921040Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}]2289[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2290[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2291[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2292[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2293[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2294[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2295[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2296[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2297[{'id': 5, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-10-02T14:16:18.883Z', 'started_at': None, 'finished_at': '2026-10-02T14:16:28.406Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '45e21e5de49e68885eab25da3122e70bb390a5df', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-10-02T14:16:28.616Z', 'started_at': None, 'finished_at': '2026-10-02T14:16:28.804Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]