vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #378
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.cpbHZCaJ0u', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: e691e69c-ea7b-4585-98e2-4db10292e37f16(finished: start all VMs, in 70.34 seconds)17gitlab # Superblock backups stored on blocks:18gitlab: waiting for unit setup-gitlab.service19gitlab # 32768, 98304, 163840, 22937620gitlab: waiting for the VM to finish booting21gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/iypr4k843dfbqwvgx5yylr1m93w2g1ny-nixos-system-gitlab-test/init regInfo=/nix/store/ypzpmwh1m4g656941axjpn9a3qk9kg4h-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/ypzpmwh1m4g656941axjpn9a3qk9kg4h-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000030] arm-pv: using stolen time PV108gitlab # [ 0.000367] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000538] Console: colour dummy device 80x25110gitlab # [ 0.000545] printk: legacy console [tty0] enabled111gitlab # [ 0.000721] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000727] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000810] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000943] landlock: Up and running.115gitlab # [ 0.000946] Yama: becoming mindful.116gitlab # [ 0.001376] LSM support for eBPF active117gitlab # [ 0.001545] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001660] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002845] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003527] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003531] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003685] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004676] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.004769] EFI services will not be available.125gitlab # [ 0.004960] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005631] Detected PIPT I-cache on CPU1127gitlab # [ 0.005730] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005861] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000129gitlab # [ 0.005986] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007088] Detected PIPT I-cache on CPU2131gitlab # [ 0.007199] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007336] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000133gitlab # [ 0.007465] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008545] Detected PIPT I-cache on CPU3135gitlab # [ 0.008663] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008795] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000137gitlab # [ 0.008924] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.009411] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.009436] SMP: Total of 4 processors activated.140gitlab # [ 0.009439] CPU: All CPU(s) started at EL1141gitlab # [ 0.009447] CPU features: detected: Branch Target Identification142gitlab # [ 0.009451] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.009454] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.009457] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.009460] CPU features: detected: Common not Private translations146gitlab # [ 0.009463] CPU features: detected: CRC32 instructions147gitlab # [ 0.009465] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.009469] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.009472] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.009474] CPU features: detected: E0PD151gitlab # [ 0.009476] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.009479] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.009482] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.009485] CPU features: detected: Fine Grained Traps155gitlab # [ 0.009488] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.009492] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.009495] CPU features: detected: LSE atomic instructions158gitlab # [ 0.009497] CPU features: detected: Privileged Access Never159gitlab # [ 0.009500] CPU features: detected: PMUv3160gitlab # [ 0.009502] CPU features: detected: RAS Extension Support161gitlab # [ 0.009504] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.009507] CPU features: detected: Random Number Generator163gitlab # [ 0.009509] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.009512] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.009514] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.009518] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.009648] alternatives: applying system-wide alternatives168gitlab # [ 0.012542] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.012788] Memory: 5963780K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.014488] devtmpfs: initialized171gitlab # [ 0.018504] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.018570] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.018745] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.018749] 0 pages in range for non-PLT usage175gitlab # [ 0.018751] 508288 pages in range for PLT usage176gitlab # [ 0.018868] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.019666] DMI not present or invalid.178gitlab # [ 0.022873] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.025634] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.025907] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.027764] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.027784] audit: initializing netlink subsys (disabled)183gitlab # [ 0.028085] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.028599] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.028602] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.028606] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.028609] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.028612] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.028698] cpuidle: using governor ladder190gitlab # [ 0.028762] cpuidle: using governor menu191gitlab # [ 0.028951] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.029156] ASID allocator initialised with 65536 entries193gitlab # [ 0.030343] Serial: AMBA PL011 UART driver194gitlab # [ 0.035474] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.035771] printk: console [ttyAMA0] enabled196gitlab # [ 0.046522] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.046528] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.046531] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.046534] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.046537] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.046539] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.046542] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.046545] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.060079] fbcon: Taking over console205gitlab # [ 0.060110] ACPI: Interpreter disabled.206gitlab # [ 0.062729] iommu: Default domain type: Translated207gitlab # [ 0.062737] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.077992] SCSI subsystem initialized209gitlab # [ 0.078459] usbcore: registered new interface driver usbfs210gitlab # [ 0.078500] usbcore: registered new interface driver hub211gitlab # [ 0.078523] usbcore: registered new device driver usb212gitlab # [ 0.078904] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.078909] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.078920] PTP clock support registered215gitlab # [ 0.078980] EDAC MC: Ver: 3.0.0216gitlab # [ 0.079258] scmi_core: SCMI protocol bus registered217gitlab # [ 0.079880] FPGA manager framework218gitlab # [ 0.080945] vgaarb: loaded219gitlab # [ 0.084264] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.084861] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.084890] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.086465] netfs: FS-Cache loaded223gitlab # [ 0.087168] pnp: PnP ACPI: disabled224gitlab # [ 0.098091] NET: Registered PF_INET protocol family225gitlab # [ 0.100853] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.156485] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.156551] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.156609] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.157020] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.157852] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.158033] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.158112] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.158286] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.158581] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.158701] NET: Registered PF_XDP protocol family236gitlab # [ 0.158720] PCI: CLS 0 bytes, default 64237gitlab # [ 0.158965] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.177773] kvm [1]: HYP mode not available239gitlab # [ 0.249607] Initialise system trusted keyrings240gitlab # [ 0.249907] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.250714] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.250856] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.266104] Key type asymmetric registered244gitlab # [ 0.266109] Asymmetric key parser 'x509' registered245gitlab # [ 0.266182] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.266465] io scheduler mq-deadline registered247gitlab # [ 0.266472] io scheduler kyber registered248gitlab # [ 0.278473] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.280410] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.281039] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.281059] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.281107] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.281117] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.281145] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.281173] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.281289] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.281300] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.281306] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.281310] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.281315] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.281413] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.282330] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.282671] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.282690] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.282732] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.282751] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.283569] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.283915] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.283932] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.283973] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.284804] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.285158] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.285176] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.285216] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.286017] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.286368] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.286386] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.286427] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.287270] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.287611] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.287628] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.287669] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.288471] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.288812] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.288830] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.288879] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.289706] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.290041] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.290059] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.290107] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.290126] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.290945] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.291291] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.291332] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.292129] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.292488] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.292536] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.293216] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.293542] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.293999] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.294425] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.294443] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.294484] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.295292] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.295621] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.295639] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.295679] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.296731] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.296741] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.296746] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.296814] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.296891] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.296958] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.297025] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.297104] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.297186] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.297263] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.297337] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.297414] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.297513] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.297595] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.297623] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.297650] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.297677] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.297705] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.297732] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.297760] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.297788] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.297817] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.297845] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.297873] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.297909] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.297943] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.297971] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.298008] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.298036] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.298063] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.298090] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.298126] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.298154] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.298183] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.298216] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.298223] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.298228] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.300465] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.317694] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.320866] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.323754] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.326751] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.332585] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.335396] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.338102] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.340965] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.347243] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.349784] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.353013] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.360838] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.363475] msm_serial: driver initialized359gitlab # [ 0.363653] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.363716] STM32 USART driver initialized361gitlab # [ 0.390485] loop: module loaded362gitlab # [ 0.390707] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.393004] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.395535] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.397677] virtio_blk virtio5: [vdb] 12285264 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.400150] megasas: 07.734.00.00-rc1367gitlab # [ 0.401024] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.403736] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.403806] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.405895] Using buffer write method371gitlab # [ 0.405930] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.408103] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.408120] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.410285] Using buffer write method375gitlab # [ 0.410301] Concatenating MTD devices:376gitlab # [ 0.410304] (0): "0.flash"377gitlab # [ 0.410307] (1): "0.flash"378gitlab # [ 0.410309] into device "0.flash"379gitlab # [ 0.576422] Freeing initrd memory: 27184K380gitlab # [ 0.593805] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.602485] thunder_xcv, ver 1.0382gitlab # [ 0.602624] thunder_bgx, ver 1.0383gitlab # [ 0.602684] nicpf, ver 1.0384gitlab # [ 0.604363] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.604375] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.604440] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.604454] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.604522] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.604531] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.604616] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.604626] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.605024] sky2: driver version 1.30393gitlab # [ 0.609744] usbcore: registered new interface driver usb-storage394gitlab # [ 0.610504] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.610561] usbserial: USB Serial support registered for generic396gitlab # [ 0.610608] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.610690] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.611487] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.612493] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.615058] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.615126] rtc-pl031 9010000.pl031: setting system clock to 2026-09-13T12:59:37 UTC (1789304377)402gitlab # [ 0.615973] i2c_dev: i2c /dev entries driver403gitlab # [ 0.620391] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00404gitlab # [ 0.621081] hub 1-0:1.0: USB hub found405gitlab # [ 0.621104] hub 1-0:1.0: 6 ports detected406gitlab # [ 0.624183] sdhci: Secure Digital Host Controller Interface driver407gitlab # [ 0.624198] sdhci: Copyright(c) Pierre Ossman408gitlab # [ 0.625069] Synopsys Designware Multimedia Card Interface Driver409gitlab # [ 0.626212] sdhci-pltfm: SDHCI platform and OF driver helper410gitlab # [ 0.631029] hid: raw HID events driver (C) Jiri Kosina411gitlab # [ 0.631949] usbcore: registered new interface driver usbhid412gitlab # [ 0.631960] usbhid: USB HID core driver413gitlab # [ 0.663577] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.668358] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.668797] NET: Registered PF_INET6 protocol family416gitlab # [ 0.670587] Segment Routing with IPv6417gitlab # [ 0.670623] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.670748] NET: Registered PF_PACKET protocol family419gitlab # [ 0.671164] 9pnet: Installing 9P2000 support420gitlab # [ 0.675578] Key type dns_resolver registered421gitlab # [ 0.693987] registered taskstats version 1422gitlab # [ 0.694519] Loading compiled-in X.509 certificates423gitlab # [ 0.719104] Demotion targets for Node 0: null424gitlab # [ 0.719379] Key type .fscrypt registered425gitlab # [ 0.719386] Key type fscrypt-provisioning registered426gitlab # [ 0.719600] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.719626] ima: Allocated hash algorithm: sha1428gitlab # [ 0.719658] ima: No architecture policies found429gitlab # [ 0.721094] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.751429] clk: Disabling unused clocks431gitlab # [ 0.751460] PM: genpd: Disabling unused power domains432gitlab # [ 0.756911] Freeing unused kernel memory: 4736K433gitlab # [ 0.757150] Run /init as init process434gitlab # [ 0.791006] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.884381] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 1.041117] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.125591] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.125659] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.125726] systemd[1]: Detected architecture arm64.440gitlab # [ 1.125741] systemd[1]: Running in initrd.441gitlab # [ 1.126597] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.126854] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.138096] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.256332] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.340595] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.412082] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.414391] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2448gitlab # [ 1.414845] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0449gitlab # [ 1.439019] systemd[1]: Created slice Slice /system/modprobe.450gitlab # [ 1.439268] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.451gitlab # [ 1.439303] systemd[1]: Expecting device /dev/disk/by-label/nix-store...452gitlab # [ 1.439318] systemd[1]: Expecting device /dev/disk/by-label/nixos...453gitlab # [ 1.439338] systemd[1]: Reached target Path Units.454gitlab # [ 1.439356] systemd[1]: Reached target Slice Units.455gitlab # [ 1.439372] systemd[1]: Reached target Swaps.456gitlab # [ 1.439389] systemd[1]: Reached target Timer Units.457gitlab # [ 1.439548] systemd[1]: Listening on D-Bus System Message Bus Socket.458gitlab # [ 1.439707] systemd[1]: Listening on Journal Socket (/dev/log).459gitlab # [ 1.439866] systemd[1]: Listening on Journal Sockets.460gitlab # [ 1.440008] systemd[1]: Listening on udev Control Socket.461gitlab # [ 1.440091] systemd[1]: Listening on udev Kernel Socket.462gitlab # [ 1.440110] systemd[1]: Reached target Socket Units.463gitlab # [ 1.441898] systemd[1]: Starting Create List of Static Device Nodes...464gitlab # [ 1.443672] systemd[1]: Starting Load Kernel Module 9pnet_virtio...465gitlab # [ 1.443744] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs466gitlab # [ 1.445342] systemd[1]: Mounting Kernel Configuration File System...467gitlab # [ 1.472778] systemd[1]: Starting Journal Service...468gitlab # [ 1.474654] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.474748] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.476087] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.477791] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.478738] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.479178] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.479528] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.492481] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...476gitlab # [ 1.524578] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.477gitlab # [ 1.527183] systemd-journald[94]: Collecting audit messages is disabled.478gitlab # [ 1.527876] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.479gitlab # [ 1.528207] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0480gitlab # [ 1.528531] systemd[1]: Starting Create Static Device Nodes in /dev...481gitlab # [ 1.529072] [drm] features: -virgl +edid -resource_blob -host_visible482gitlab # [ 1.529080] [drm] features: -context_init483gitlab # [ 1.529624] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev484gitlab # [ 1.541414] [drm] number of scanouts: 1485gitlab # [ 1.541480] [drm] number of cap sets: 0486gitlab # [ 1.542480] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic487gitlab # [ 1.542504] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0488gitlab # [ 1.546054] systemd[1]: Finished Create Static Device Nodes in /dev.489gitlab # [ 1.546221] systemd[1]: Reached target Preparation for Local File Systems.490gitlab # [ 1.546240] systemd[1]: Reached target Local File Systems.491gitlab # [ 1.557871] Console: switching to colour frame buffer device 160x50492gitlab # [ 1.559051] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device493gitlab # [ 1.564399] systemd[1]: Starting Rule-based Manager for Device Events and Files...494gitlab # [ 1.572787] systemd[1]: Finished Load Kernel Modules.495gitlab # [ 1.596545] systemd[1]: Starting Apply Kernel Variables...496gitlab # [ 1.612548] systemd-modules-load[95]: Using 4 probe threads497gitlab # [ 1.618136] systemd[1]: Started Journal Service.498gitlab # [ 1.616624] systemd-modules-load[95]: Module 'virtio_balloon' is built in499gitlab # [ 1.618067] systemd-modules-load[95]: Module 'virtio_console' is built in500gitlab # [ 1.619442] systemd-modules-load[95]: Module 'virtio_rng' is built in501gitlab # [ 1.620807] systemd-modules-load[95]: Inserted module 'dm_mod'502gitlab # [ 1.622041] systemd-modules-load[95]: Inserted module 'virtio_gpu'503gitlab # [ 1.623796] systemd[1]: Starting Create System Files and Directories...504gitlab # [ 1.627064] systemd-udevd[103]: Using default interface naming scheme 'v261'.505gitlab # [ 1.636172] systemd[1]: Finished Apply Kernel Variables.506gitlab # [ 1.650939] systemd[1]: Finished Create System Files and Directories.507gitlab # [ 1.653264] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.675746] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.735314] systemd-vconsole-setup[129]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.738784] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.082113] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.084666] systemd[1]: Reached target System Initialization.513gitlab # [ 2.087099] systemd[1]: Reached target Basic System.514gitlab # [ 2.287721] (udev-worker)[125]: Network interface NamePolicy= disabled on kernel command line.515gitlab # [ 2.294188] (udev-worker)[145]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.516gitlab # [ 2.301516] (udev-worker)[145]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.321730] systemd[1]: Found device /dev/disk/by-label/nixos.518gitlab # [ 2.328541] systemd[1]: Found device /dev/disk/by-label/nix-store.519gitlab # [ 2.330838] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.332438] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.373968] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.379426] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.476285] systemd[1]: Mounting /sysroot...524gitlab # [ 2.547168] EXT4-fs (vda): mounted filesystem e691e69c-ea7b-4585-98e2-4db10292e37f r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.545449] systemd[1]: Mounted /sysroot.526gitlab # [ 2.548005] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.552284] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.558052] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.568560] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.574395] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.576879] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.587595] systemd[1]: Starting Mountpoints Configured in the Real Root...533gitlab # [ 2.591128] systemd[1]: Mounted /sysroot/nix/.rw-store.534gitlab # [ 2.592541] systemd[1]: Mounted /sysroot/run.535gitlab # [ 2.613342] systemd-sysroot-fstab-check[165]: /sysroot should be mounted in the initrd, will request daemon-reload.536gitlab # [ 2.624399] systemd[1]: Starting rw-sysroot-nix-store.service...537gitlab # [ 2.630981] systemd[1]: Reload requested from client PID 165 ('systemd-sysroot') (unit initrd-parse-etc.service)...538gitlab # [ 2.632632] systemd[1]: Reloading...539gitlab # [ 2.797206] erofs (device vdb): mounted with root inode @ nid 128.540gitlab # [ 2.854657] systemd[1]: Reloading finished in 222 ms.541gitlab # [ 2.905156] systemd-sysroot-fstab-check[165]: Requesting initrd-fs.target/start/replace...542gitlab # [ 2.907392] systemd[1]: Mounted /sysroot/nix/.ro-store.543gitlab # [ 2.908303] systemd[1]: Mounted /sysroot/tmp/shared.544gitlab # [ 2.909358] systemd[1]: Mounted /sysroot/tmp/xchg.545gitlab # [ 2.910124] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.546gitlab # [ 2.911505] systemd[1]: Finished rw-sysroot-nix-store.service.547gitlab # [ 2.913346] systemd-sysroot-fstab-check[165]: Requesting swap.target/start/replace...548gitlab # [ 2.917293] systemd[1]: initrd-parse-etc.service: Deactivated successfully.549gitlab # [ 2.920152] systemd[1]: Finished Mountpoints Configured in the Real Root.550gitlab # [ 2.921203] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.551gitlab # [ 2.924057] systemd[1]: Starting rw-sysroot-nix-store.service...552gitlab # [ 2.947941] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.553gitlab # [ 2.949115] systemd[1]: Finished rw-sysroot-nix-store.service.554gitlab # [ 2.950661] (udev-worker)[137]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory555gitlab # [ 2.952964] (udev-worker)[137]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory556gitlab # [ 2.964254] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.557gitlab # [ 2.966067] systemd[1]: Stopped Virtual Console Setup.558gitlab # [ 2.966978] systemd[1]: Stopping Virtual Console Setup...559gitlab # [ 2.967769] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 3.005511] systemd-vconsole-setup[198]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 3.007739] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 3.480685] systemd[1]: Mounting /sysroot/nix/store...563gitlab # [ 3.546962] systemd[1]: Mounted /sysroot/nix/store.564gitlab # [ 3.549366] systemd[1]: Reached target Initrd File Systems.565gitlab # [ 3.551790] systemd[1]: Starting Find NixOS closure...566gitlab # [ 3.554586] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...567gitlab # [ 3.603803] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.568gitlab # [ 3.607509] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.569gitlab # [ 3.623817] systemd[1]: Finished Find NixOS closure.570gitlab # [ 3.626401] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 3.628977] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 3.676111] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 3.678649] systemd[1]: Stopped target Basic System.574gitlab # [ 3.680996] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 3.683373] systemd[1]: Stopped target Path Units.576gitlab # [ 3.685586] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 3.688913] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 3.692201] systemd[1]: Stopped target Slice Units.579gitlab # [ 3.694385] systemd[1]: Stopped target Socket Units.580gitlab # [ 3.696611] systemd[1]: Stopped target System Initialization.581gitlab # [ 3.699081] systemd[1]: Stopped target Swaps.582gitlab # [ 3.702546] systemd[1]: Stopped target Timer Units.583gitlab # [ 3.704904] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 3.707437] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 3.710220] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 3.716828] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 3.719133] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 3.721918] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 3.724334] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 3.726720] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 3.728837] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 3.731549] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 3.733449] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 3.736296] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 3.739024] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 3.741783] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 3.744005] systemd[1]: Stopped target Local File Systems.598gitlab # [ 3.746110] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 3.748354] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 3.750739] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 3.752522] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 3.754788] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 3.757015] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 3.758482] systemd[1]: initrd-cleanup.service: Deactivated successfully.605gitlab # [ 3.760273] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.606gitlab # [ 3.762001] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.607gitlab # [ 3.763776] systemd[1]: Finished Load Kernel Module 9pnet_virtio.608gitlab # [ 3.765435] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 3.767212] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 3.768680] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 3.770257] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 3.771855] systemd[1]: systemd-udevd.service: Consumed 2.391s CPU time over 2.194s wall clock time, 31.6M memory peak.613gitlab # [ 3.774267] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 3.775794] systemd[1]: Closed udev Control Socket.615gitlab # [ 3.776952] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 3.778127] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 3.779727] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 3.781258] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 3.782867] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 3.784448] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 3.785793] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 3.863994] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 3.867274] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 3.869666] systemd[1]: Reached target Switch Root.625gitlab # [ 3.871817] systemd[1]: Starting NixOS Activation...626gitlab # [ 4.042625] initrd-nixos-activation-start[223]: booting system configuration /nix/store/iypr4k843dfbqwvgx5yylr1m93w2g1ny-nixos-system-gitlab-test627gitlab # [ 4.104079] initrd-nixos-activation-start[223]: running activation script...628gitlab # [ 4.413349] initrd-nixos-activation-start[246]: setting up /etc...629gitlab # [ 4.542403] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.630gitlab # [ 4.545947] systemd[1]: Finished NixOS Activation.631gitlab # [ 4.548875] systemd[1]: Starting Switch Root...632gitlab # [ 4.590593] systemd[1]: Switching root.633gitlab # [ 4.708827] systemd-journald[94]: Received SIGTERM from PID 1 (systemd).634gitlab # [ 5.349466] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)635gitlab # [ 5.349615] systemd[1]: Detected virtualization qemu.636gitlab # [ 5.349691] systemd[1]: Detected architecture arm64.637gitlab # [ 5.349864] systemd[1]: Detected first boot.638gitlab # [ 5.351221] systemd[1]: Initializing machine ID from random generator.639gitlab # [ 5.604604] systemd[1]: bpf-restrict-fs: LSM BPF program attached640gitlab # [ 5.730703] systemd[1]: Applying preset policy.641gitlab # [ 5.907039] systemd[1]: Populated /etc with preset unit settings.642gitlab # [ 6.055338] systemd[1]: initrd-switch-root.service: Deactivated successfully.643gitlab # [ 6.055579] systemd[1]: Stopped initrd-switch-root.service.644gitlab # [ 6.057201] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.645gitlab # [ 6.065003] systemd[1]: Created slice Slice /system/getty.646gitlab # [ 6.068852] systemd[1]: Created slice GitLab DevOps Platform Slice.647gitlab # [ 6.073413] systemd[1]: Created slice User and Session Slice.648gitlab # [ 6.074136] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.649gitlab # [ 6.075576] systemd[1]: Started Forward Password Requests to Wall Directory Watch.650gitlab # [ 6.076766] systemd[1]: Expecting device /dev/hvc0...651gitlab # [ 6.077827] systemd[1]: Expecting device /dev/ttyAMA0...652gitlab # [ 6.078918] systemd[1]: Reached target Local Encrypted Volumes.653gitlab # [ 6.079954] systemd[1]: Stopped target initrd-fs.target.654gitlab # [ 6.080982] systemd[1]: Stopped target initrd-root-fs.target.655gitlab # [ 6.081985] systemd[1]: Stopped target initrd-switch-root.target.656gitlab # [ 6.082998] systemd[1]: Reached target Virtual Machines and Containers.657gitlab # [ 6.084020] systemd[1]: Reached target Path Units.658gitlab # [ 6.085055] systemd[1]: Reached target Remote File Systems.659gitlab # [ 6.086040] systemd[1]: Reached target Slice Units.660gitlab # [ 6.087053] systemd[1]: Reached target Swaps.661gitlab # [ 6.090706] systemd[1]: Listening on Query the User Interactively for a Password.662gitlab # [ 6.094803] systemd[1]: Listening on Process Core Dump Socket.663gitlab # [ 6.097782] systemd[1]: Listening on Credential Encryption/Decryption.664gitlab # [ 6.101369] systemd[1]: Listening on Factory Reset Management.665gitlab # [ 6.102741] systemd[1]: Listening on Hostname Service Socket.666gitlab # [ 6.111939] systemd[1]: Starting Journal Log Access Socket...667gitlab # [ 6.114241] systemd[1]: Listening on Journal Audit Socket.668gitlab # [ 6.118550] systemd[1]: Listening on Console Output Muting Service Socket.669gitlab # [ 6.120267] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.670gitlab # [ 6.121605] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os671gitlab # [ 6.122710] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki672gitlab # [ 6.129201] systemd[1]: Listening on Disk Repartitioning Service Socket.673gitlab # [ 6.130684] systemd[1]: Listening on udev Control Socket.674gitlab # [ 6.132095] systemd[1]: Listening on udev Varlink Socket.675gitlab # [ 6.172921] systemd[1]: Mounting Huge Pages File System...676gitlab # [ 6.178638] systemd[1]: Mounting POSIX Message Queue File System...677gitlab # [ 6.184360] systemd[1]: Mounting Kernel Debug File System...678gitlab # [ 6.193171] systemd[1]: Mounting Kernel Trace File System...679gitlab # [ 6.201456] systemd[1]: Starting Create List of Static Device Nodes...680gitlab # [ 6.212695] systemd[1]: Starting Load Kernel Module 9pnet_virtio...681gitlab # [ 6.214038] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs682gitlab # [ 6.224527] systemd[1]: Mounting Kernel Configuration File System...683gitlab # [ 6.225930] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm684gitlab # [ 6.228557] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore685gitlab # [ 6.240536] systemd[1]: Starting Load Kernel Module fuse...686gitlab # [ 6.241457] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67687gitlab # [ 6.260542] systemd[1]: Starting Journal Service...688gitlab # [ 6.264800] systemd[1]: Starting Load Kernel Modules...689gitlab # [ 6.268822] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...690gitlab # [ 6.287841] systemd[1]: Starting Remount Root and Kernel File Systems...691gitlab # [ 6.288596] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os692gitlab # [ 6.302927] systemd[1]: Starting Coldplug All udev Devices...693gitlab # [ 6.303626] fuse: init (API version 7.45)694gitlab # [ 6.303728] systemd-journald[317]: Collecting audit messages is enabled.695gitlab # [ 6.314818] systemd[1]: Listening on Journal Log Access Socket.696gitlab # [ 6.315714] systemd[1]: Mounted Huge Pages File System.697gitlab # [ 6.316652] systemd[1]: Mounted POSIX Message Queue File System.698gitlab # [ 6.318623] systemd[1]: Mounted Kernel Debug File System.699gitlab # [ 6.319355] systemd[1]: Mounted Kernel Trace File System.700gitlab # [ 6.320577] systemd[1]: Finished Create List of Static Device Nodes.701gitlab # [ 6.321690] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.702gitlab # [ 6.326271] systemd[1]: Queued start job for default target Multi-User System.[ 6.322670] systemd[1]: Finished Load Kernel Module 9pnet_virtio.703gitlab # 704gitlab # [ 6.323569] systemd[1]: Mounted Kernel Configuration File System.705gitlab # [ 6.324491] systemd[1]: modprobe@fuse.service: Deactivated successfully.706gitlab # [ 6.325479] systemd[1]: Finished Load Kernel Module fuse.707gitlab # [ 6.326601] systemd[1]: Finished Load Kernel Modules.708gitlab # [ 6.341530] systemd[1]: Unnecessary job was removed for /dev/vdb.[ 6.336463] systemd[1]: Mounting FUSE Control File System...709gitlab # [ 6.350768] EXT4-fs (vda): re-mounted e691e69c-ea7b-4585-98e2-4db10292e37f.710gitlab # 711gitlab # [ 6.352999] systemd[1]: Starting Firewall...712gitlab # [ 6.356644] systemd[1]: Starting Apply Kernel Variables...713gitlab # [ 6.358385] systemd[1]: systemd-journald.service: Deactivated successfully.714gitlab # [ 6.359674] systemd-modules-load[318]: Using 3 probe threads[ 6.365021] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...715gitlab # 716gitlab # [ 6.366915] systemd[1]: Started Journal Service.717gitlab # [ 6.371820] systemd-modules-load[318]: Module 'atkbd' is built in718gitlab # [ 6.372999] systemd-modules-load[318]: Module 'loop' is built in719gitlab # [ 6.376380] systemd-modules-load[318]: Inserted module 'tls'720gitlab # [ 6.379162] systemd[1]: Finished Remount Root and Kernel File Systems.721gitlab # [ 6.381348] systemd[1]: Mounted FUSE Control File System.722gitlab # [ 6.384239] systemd[1]: Finished Apply Kernel Variables.723gitlab # [ 6.385461] systemd-oomd[320]: No swap; memory pressure usage will be degraded724gitlab # [ 6.388791] systemd[1]: Listening on Disk Image Download Service Socket.725gitlab # [ 6.392496] systemd[1]: Starting Flush Journal to Persistent Storage...726gitlab # [ 6.394603] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore727gitlab # [ 6.399187] systemd[1]: Starting Load/Save OS Random Seed...728gitlab # [ 6.401836] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os729gitlab # [ 6.405242] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.730gitlab # [ 6.416881] systemd-journald[317]: Received client request to flush runtime journal.731gitlab # [ 6.447020] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.732gitlab # [ 6.449887] systemd[1]: Finished Load/Save OS Random Seed.733gitlab # [ 6.451138] systemd[1]: Reached target First Boot Complete.734gitlab # [ 6.452393] systemd[1]: Starting Create Static Device Nodes in /dev...735gitlab # [ 6.453640] systemd[1]: Finished Create Static Device Nodes in /dev.736gitlab # [ 6.454542] systemd[1]: Reached target Preparation for Local File Systems.737gitlab # [ 6.455468] systemd[1]: Starting Rule-based Manager for Device Events and Files...738gitlab # [ 6.456562] systemd[1]: Finished Flush Journal to Persistent Storage.739gitlab # [ 6.514244] systemd-udevd[363]: Using default interface naming scheme 'v261'.740gitlab # [ 6.549226] systemd[1]: Started Rule-based Manager for Device Events and Files.741gitlab # [ 6.837501] systemd[1]: Finished Firewall.742gitlab # [ 6.929769] systemd[1]: Finished Coldplug All udev Devices.743gitlab # [ 7.002418] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs744gitlab # [ 7.064174] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.745gitlab # [ 7.072443] systemd[1]: Mounting /run/wrappers...746gitlab # [ 7.076546] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.747gitlab # [ 7.102569] (udev-worker)[410]: Network interface NamePolicy= disabled on kernel command line.748gitlab # [ 7.122180] systemd[1]: Mounted /run/wrappers.749gitlab # [ 7.134846] systemd[1]: Reached target Local File Systems.750gitlab # [ 7.138547] systemd[1]: Listening on Boot Loader Control Service Socket.751gitlab # [ 7.140999] (udev-worker)[472]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.752gitlab # [ 7.145194] (udev-worker)[472]: Network interface NamePolicy= disabled on kernel command line.753gitlab # [ 7.147065] systemd[1]: Starting register-nix-paths.service...754gitlab # [ 7.157571] systemd[1]: Starting Create SUID/SGID Wrappers...755gitlab # [ 7.158490] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.756gitlab # [ 7.161391] systemd[1]: Starting Save Transient machine-id to Disk...757gitlab # [ 7.172416] systemd[1]: Starting Create System Files and Directories...758gitlab # [ 7.173978] systemd[1]: Condition check resulted in Virtio network device being skipped.759gitlab # [ 7.175793] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore760gitlab # [ 7.184157] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.761gitlab # [ 7.185678] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67762gitlab # [ 7.189774] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore763gitlab # [ 7.191406] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os764gitlab # [ 7.193225] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os765gitlab # [ 7.213203] mousedev: PS/2 mouse device common for all mice766gitlab # [ 7.215003] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.767gitlab # [ 7.216996] systemd[1]: Finished Save Transient machine-id to Disk.768gitlab # [ 7.222848] systemd[1]: Finished Create System Files and Directories.769gitlab # [ 7.231202] systemd[1]: Starting Rebuild Journal Catalog...770gitlab # [ 7.235208] systemd[1]: Starting Record System Boot/Shutdown in UTMP...771gitlab # [ 7.263147] systemd[1]: Finished Record System Boot/Shutdown in UTMP.772gitlab # [ 7.276451] systemd[1]: Finished Rebuild Journal Catalog.773gitlab # [ 7.283324] systemd[1]: Starting Update is Completed...774gitlab # [ 7.311257] systemd[1]: Finished Update is Completed.775gitlab # [ 7.340431] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.776gitlab # [ 7.343257] systemd[1]: Finished Create SUID/SGID Wrappers.777gitlab # [ 7.344588] systemd[1]: suid-sgid-wrappers.service: Consumed 137ms CPU time over 185ms wall clock time, 6.2M memory peak, 1.9M read from disk.778gitlab # [ 7.612218] systemd[1]: Finished register-nix-paths.service.779gitlab # [ 7.613191] systemd[1]: Reached target System Initialization.780gitlab # [ 7.614776] systemd[1]: Started Discard unused filesystem blocks once a week.781gitlab # [ 7.616979] systemd[1]: Started logrotate.timer.782gitlab # [ 7.618975] systemd[1]: Started Daily Cleanup of Temporary Directories.783gitlab # [ 7.620478] systemd[1]: Reached target Timer Units.784gitlab # [ 7.623087] systemd[1]: Listening on D-Bus System Message Bus Socket.785gitlab # [ 7.624260] systemd[1]: Listening on Nix Daemon Socket.786gitlab # [ 7.636106] systemd[1]: Starting nixbot CI service socket...787gitlab # [ 7.637031] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.788gitlab # [ 7.641549] systemd[1]: Starting D-Bus System Message Bus...789gitlab # [ 7.642477] systemd[1]: Listening on nixbot CI service socket.790gitlab # [ 7.643326] systemd[1]: Reached target Socket Units.791gitlab # [ 7.677085] dbus-broker-launch[564]: Looking up NSS user entry for 'systemd-timesync'...792gitlab # [ 7.678933] dbus-broker-launch[564]: NSS returned no entry for 'systemd-timesync'793gitlab # [ 7.680466] dbus-broker-launch[564]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"794gitlab # [ 7.683568] systemd[1]: Started D-Bus System Message Bus.795gitlab # [ 7.703768] dbus-broker-launch[564]: Ready796gitlab # [ 7.705458] systemd[1]: Reached target Basic System.797gitlab # [ 7.710228] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3798gitlab # [ 7.716409] systemd[1]: Started backdoor.service.799gitlab # [ 7.742003] systemd[1]: Starting gitlab-config.service...800gitlab # [ 7.744229] systemd[1]: Starting Import lastlog data into lastlog2 database...801gitlab # [ 7.747552] systemd[1]: Starting Logrotate configuration check...802gitlab # connecting to host...803gitlab # [ 7.764940] systemd[1]: Starting Name Service Cache Daemon (nsncd)...804gitlab # [ 7.768201] systemd[1]: Starting Post-Boot Actions...805gitlab # [ 7.772308] systemd[1]: Started Reset console on configuration changes.806gitlab # [ 7.776138] logrotate[574]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.807gitlab # [ 7.779369] logrotate[574]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf808gitlab # [ 7.780746] logrotate[574]: note: 'monthly' overrides previously specified 'weekly'809gitlab: Guest shell says: b'Spawning backdoor root shell...\n'810gitlab # [ 7.781772] logrotate[574]: Reading state from file: /var/lib/logrotate.status811gitlab # [ 7.788058] logrotate[574]: state file /var/lib/logrotate.status does not exist812gitlab # [ 7.788998] logrotate[574]: Allocating hash table for state file, size 64 entries813gitlab: connected to guest root shell814gitlab # [ 7.789949] logrotate[574]: Handling 4 logs815gitlab: (connecting took 8.26 seconds)816gitlab # [ 7.790542] logrotate[574]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed817gitlab: (finished: waiting for the VM to finish booting, in 8.26 seconds)818gitlab # [ 7.797411] logrotate[574]: considering log /var/log/btmp819gitlab # [ 7.798214] logrotate[574]: Creating new state820gitlab # [ 7.798777] logrotate[574]: Now: 2026-09-13 12:59821gitlab # [ 7.799514] logrotate[574]: Last rotated at 2026-09-13 12:00822gitlab # [ 7.801286] logrotate[574]: log does not need rotating (log has already been rotated)823gitlab # [ 7.802354] logrotate[574]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed824gitlab # [ 7.809084] logrotate[574]: considering log /var/log/wtmp825gitlab # [ 7.809851] logrotate[574]: Creating new state826gitlab # [ 7.810457] logrotate[574]: Now: 2026-09-13 12:59827gitlab # [ 7.811115] logrotate[574]: Last rotated at 2026-09-13 12:00828gitlab # [ 7.811915] logrotate[574]: log does not need rotating (log has already been rotated)829gitlab # [ 7.813062] logrotate[574]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed830gitlab # [ 7.814858] logrotate[574]: switching euid from 0 to 165 and egid from 0 to 165 (pid 574)831gitlab # [ 7.815878] logrotate[574]: considering log /var/gitlab/state/log/*.log832gitlab # [ 7.817001] logrotate[574]: log /var/gitlab/state/log/*.log does not exist -- skipping833gitlab # [ 7.818101] logrotate[574]: Creating new state834gitlab # [ 7.818690] logrotate[574]: switching euid from 165 to 0 and egid from 165 to 0 (pid 574)835gitlab # [ 7.819926] logrotate[574]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed836gitlab # [ 7.822088] logrotate[574]: switching euid from 0 to 60 and egid from 0 to 60 (pid 574)837gitlab # [ 7.823338] logrotate[574]: considering log /var/log/nginx/*.log838gitlab # [ 7.824273] logrotate[574]: log /var/log/nginx/*.log does not exist -- skipping839gitlab # [ 7.825366] logrotate[574]: Creating new state840gitlab # [ 7.826081] logrotate[574]: not running postrotate script, since no logs were rotated841gitlab # [ 7.827132] logrotate[574]: switching euid from 60 to 0 and egid from 60 to 0 (pid 574)842gitlab # [ 7.829050] systemd[1]: Starting resolvconf update...843gitlab # [ 7.830145] nsncd[575]: Sep 13 12:59:44.685 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"844gitlab # [ 7.833141] systemd[1]: Started Name Service Cache Daemon (nsncd).845gitlab # [ 7.834822] systemd[1]: Finished Logrotate configuration check.846gitlab # [ 7.840087] systemd[1]: Finished Post-Boot Actions.847gitlab # [ 7.840937] systemd[1]: Finished Import lastlog data into lastlog2 database.848gitlab # [ 7.843811] systemd[1]: Reached target Host and Network Name Lookups.849gitlab # [ 7.844935] systemd[1]: Reached target User and Group Name Lookups.850gitlab # [ 7.848125] systemd[1]: Starting User Login Management...851gitlab # [ 7.852585] systemd[1]: Starting Virtual Console Setup...852gitlab # [ 7.874574] systemd[1]: Stopped target Host and Network Name Lookups.853gitlab # [ 7.877492] systemd[1]: Stopping Host and Network Name Lookups...854gitlab # [ 7.878418] systemd[1]: Stopped target User and Group Name Lookups.855gitlab # [ 7.879335] systemd[1]: Stopping User and Group Name Lookups...856gitlab # [ 7.880306] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...857gitlab # [ 7.883756] systemd[1]: nscd.service: Deactivated successfully.858gitlab # [ 7.885199] systemd[1]: Stopped Name Service Cache Daemon (nsncd).859gitlab # [ 7.886151] systemd[1]: nscd.service: Consumed 29ms CPU time over 120ms wall clock time, 7.6M memory peak, 5.8M read from disk.860gitlab # [ 7.891208] systemd[1]: Starting Name Service Cache Daemon (nsncd)...861gitlab # [ 7.912691] systemd[1]: Started Name Service Cache Daemon (nsncd).862gitlab # [ 7.914315] nsncd[666]: Sep 13 12:59:44.801 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"863gitlab # [ 7.917723] systemd-logind[633]: New seat seat0.864gitlab # [ 7.918516] systemd[1]: Reached target Host and Network Name Lookups.865gitlab # [ 7.919390] systemd[1]: Reached target User and Group Name Lookups.866gitlab # [ 7.920463] systemd-logind[633]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)867gitlab # [ 7.921716] systemd-logind[633]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)868gitlab # [ 7.922901] systemd-logind[633]: Watching system buttons on /dev/input/event0 (gpio-keys)869gitlab # [ 7.924055] systemd[1]: Started User Login Management.870gitlab # [ 7.924975] systemd[1]: Starting linger-users.service...871gitlab # [ 7.928203] systemd-vconsole-setup[637]: Configuration of first virtual console was skipped, ignoring remaining ones.872gitlab # [ 7.937085] systemd[1]: Finished Virtual Console Setup.873gitlab # [ 7.947738] systemd[1]: linger-users.service: Deactivated successfully.874gitlab # [ 7.948987] systemd[1]: Finished linger-users.service.875gitlab # [ 7.950011] systemd[1]: Finished resolvconf update.876gitlab # [ 7.952112] systemd[1]: Reached target Preparation for Network.877gitlab # [ 7.956377] systemd[1]: Starting DHCP Client...878gitlab # [ 7.958496] systemd[1]: Starting Address configuration of eth1...879gitlab # [ 7.961515] systemd[1]: Starting Extra networking commands....880gitlab # [ 8.004433] network-addresses-eth1-start[693]: adding address 192.168.1.1/24... done881gitlab # [ 8.010423] network-addresses-eth1-start[693]: adding address 2001:db8:1::1/64... done882gitlab # [ 8.017817] systemd[1]: Finished Address configuration of eth1.883gitlab # [ 8.041831] dhcpcd[703]: dhcpcd-10.3.2 starting884gitlab # [ 8.044967] dhcpcd[750]: dev: loaded udev885gitlab # [ 8.062437] 8021q: 802.1Q VLAN Support v1.8886gitlab # [ 8.062855] 8021q: adding VLAN 0 to HW filter on device eth1887gitlab # [ 8.060611] systemd[1]: Finished Extra networking commands..888gitlab # [ 8.062283] systemd[1]: Reached target Network.889gitlab # [ 8.064474] systemd[1]: Starting gitlab-workhorse.service...890gitlab # [ 8.068794] systemd[1]: Starting Nginx Web Server...891gitlab # [ 8.073980] systemd[1]: Starting PostgreSQL Server...892gitlab # [ 8.077720] systemd[1]: Starting Redis Server - redis-gitlab...893gitlab # [ 8.082697] systemd[1]: Starting Permit User Sessions...894gitlab # [ 8.090888] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.895gitlab # [ 8.101761] cfg80211: Loading compiled-in X.509 certificates for regulatory database896gitlab # [ 8.106480] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'897gitlab # [ 8.106963] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'898gitlab # [ 8.108718] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2899gitlab # [ 8.109044] cfg80211: failed to load regulatory.db900gitlab # [ 8.113572] systemd[1]: Finished Permit User Sessions.901gitlab # [ 8.119622] systemd[1]: Started Getty on tty1.902gitlab # [ 8.120528] systemd[1]: Reached target Login Prompts.903gitlab # [ 8.145614] 8021q: adding VLAN 0 to HW filter on device eth0904gitlab # [ 8.142598] dhcpcd[750]: eth0: waiting for carrier905gitlab # [ 8.145019] dhcpcd[750]: eth0: carrier acquired906gitlab # [ 8.152323] dhcpcd[750]: DUID 00:01:00:01:32:39:5a:c1:52:54:00:12:34:56907gitlab # [ 8.153431] dhcpcd[750]: eth0: IAID 00:12:34:56908gitlab # [ 8.154085] dhcpcd[750]: eth0: adding address fe80::5054:ff:fe12:3456909gitlab # [ 8.238411] nginx-pre-start[797]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok910gitlab # [ 8.240174] nginx-pre-start[797]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful911gitlab # [ 8.245621] systemd[1]: Started Nginx Web Server.912gitlab # [ 8.250058] redis-server[794]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.913gitlab # [ 8.253867] redis-server[794]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo914gitlab # [ 8.255315] systemd[1]: Started Redis Server - redis-gitlab.915gitlab # [ 8.256622] redis-server[794]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=794, just started916gitlab # [ 8.258329] redis-server[794]: Configuration loaded917gitlab # [ 8.259688] redis[794]: monotonic clock: ARM CNTVCT @ 1000 ticks/us918gitlab # [ 8.261288] redis[794]: Running mode=standalone, port=0.919gitlab # [ 8.262296] redis[794]: Server initialized920gitlab # [ 8.263109] redis[794]: Ready to accept connections unix921gitlab # [ 8.264150] redis[794]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.922gitlab # [ 8.266629] postgresql-pre-start[796]: The files belonging to this database system will be owned by user "postgres".923gitlab # [ 8.268922] postgresql-pre-start[796]: This user must also own the server process.924gitlab # [ 8.270161] postgresql-pre-start[796]: The database cluster will be initialized with locale "en_US.UTF-8".925gitlab # [ 8.271583] postgresql-pre-start[796]: The default database encoding has accordingly been set to "UTF8".926gitlab # [ 8.273995] postgresql-pre-start[796]: The default text search configuration will be set to "english".927gitlab # [ 8.275355] postgresql-pre-start[796]: Data page checksums are enabled.928gitlab # [ 8.276364] postgresql-pre-start[796]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok929gitlab # [ 8.277726] postgresql-pre-start[796]: creating subdirectories ... ok930gitlab # [ 8.278667] postgresql-pre-start[796]: selecting dynamic shared memory implementation ... posix931gitlab # [ 8.311134] postgresql-pre-start[796]: selecting default "max_connections" ... 100932gitlab # [ 8.358097] postgresql-pre-start[796]: selecting default "shared_buffers" ... 128MB933gitlab # [ 8.390525] dhcpcd[750]: eth0: soliciting a DHCP lease934gitlab # [ 8.416601] dhcpcd[750]: eth0: offered 10.0.2.15 from 10.0.2.2935gitlab # [ 8.436283] dhcpcd[750]: eth0: probing address 10.0.2.15/24936gitlab # [ 8.736539] systemd[1]: Started gitlab-workhorse.service.937gitlab # [ 8.833175] postgresql-pre-start[796]: selecting default time zone ... UTC938gitlab # [ 8.835081] postgresql-pre-start[796]: creating configuration files ... ok939gitlab # [ 8.991611] gitlab-workhorse[819]: time="2026-09-13T12:59:45Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4940gitlab # [ 8.996737] gitlab-workhorse[819]: time="2026-09-13T12:59:45Z" level=info msg="Using redis/go-redis"941gitlab # [ 8.998642] gitlab-workhorse[819]: time="2026-09-13T12:59:45Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix942gitlab # [ 9.072683] postgresql-pre-start[796]: running bootstrap script ... ok943gitlab # [ 9.472820] dhcpcd[750]: eth0: soliciting an IPv6 router944gitlab # [ 9.475225] dhcpcd[750]: eth0: Router Advertisement from fe80::2945gitlab # [ 9.478107] dhcpcd[750]: eth0: adding address fec0::5054:ff:fe12:3456/64946gitlab # [ 9.480937] dhcpcd[750]: eth0: adding route to fec0::/64947gitlab # [ 9.483255] dhcpcd[750]: eth0: adding default route via fe80::2948gitlab # [ 9.788750] postgresql-pre-start[796]: performing post-bootstrap initialization ... ok949gitlab # [ 10.068348] postgresql-pre-start[796]: syncing data to disk ... ok950gitlab # [ 10.071027] postgresql-pre-start[796]: initdb: warning: enabling "trust" authentication for local connections951gitlab # [ 10.074879] postgresql-pre-start[796]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.952gitlab # [ 10.081554] postgresql-pre-start[796]: Success. You can now start the database server using:953gitlab # [ 10.085160] postgresql-pre-start[796]: pg_ctl -D /var/lib/postgresql/18 -l logfile start954gitlab # [ 10.223448] postgres[853]: [853] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit955gitlab # [ 10.229478] postgres[853]: [853] LOG: listening on IPv6 address "::1", port 5432956gitlab # [ 10.233135] postgres[853]: [853] LOG: listening on IPv4 address "127.0.0.1", port 5432957gitlab # [ 10.237635] postgres[853]: [853] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"958gitlab # [ 10.242459] postgres[863]: [863] LOG: database system was shut down at 2026-09-13 12:59:46 GMT959gitlab # [ 10.261964] postgres[853]: [853] LOG: database system is ready to accept connections960gitlab # [ 10.266704] systemd[1]: Started PostgreSQL Server.961gitlab # [ 10.273171] systemd[1]: Starting PostgreSQL Setup Scripts...962gitlab # [ 10.647189] postgresql-setup-start[874]: CREATE DATABASE963gitlab # [ 10.690507] postgresql-setup-start[883]: CREATE ROLE964gitlab # [ 10.707307] postgresql-setup-start[885]: ALTER DATABASE965gitlab # [ 10.754001] postgresql-setup-start[891]: CREATE ROLE966gitlab # [ 10.758564] systemd[1]: Finished PostgreSQL Setup Scripts.967gitlab # [ 10.760380] systemd[1]: Reached target PostgreSQL.968gitlab # [ 10.765205] systemd[1]: Starting gitlab-postgresql.service...969gitlab # [ 10.890328] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[879]: mkdir -p /var/gitlab/state/home/.ssh: OK970gitlab # [ 10.894845] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[879]: chmod 700 /var/gitlab/state/home/.ssh: OK971gitlab # [ 10.961184] gitlab-postgresql-start[899]: CREATE DATABASE972gitlab # [ 10.977149] systemd[1]: Finished gitlab-config.service.973gitlab # [ 10.987933] systemd[1]: Started gitaly.service.974gitlab # [ 11.000445] systemd[1]: Starting gitlab-db-config.service...975gitlab # [ 11.036469] gitlab-postgresql-start[920]: CREATE EXTENSION976gitlab # [ 11.161997] gitlab-postgresql-start[929]: CREATE EXTENSION977gitlab # [ 11.165767] systemd[1]: Finished gitlab-postgresql.service.978gitlab # [ 11.714514] gitaly[919]: time="2026-09-13T12:59:48.602Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=919979gitlab # [ 11.738618] gitaly[919]: time="2026-09-13T12:59:48.602Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=919980gitlab # [ 11.743990] gitaly[919]: time="2026-09-13T12:59:48.602Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=919981gitlab # [ 11.750266] gitaly[919]: time="2026-09-13T12:59:48.602Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=919982gitlab # [ 11.759344] gitaly[919]: time="2026-09-13T12:59:48.602Z" level=info msg="Starting Gitaly" pid=919 version=983gitlab # [ 11.939885] gitaly[919]: time="2026-09-13T12:59:48.828Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=225 pid=919984gitlab # [ 11.942579] gitaly[919]: time="2026-09-13T12:59:48.829Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=919985gitlab # [ 11.945085] gitaly[919]: time="2026-09-13T12:59:48.830Z" level=info msg="finished initializing command factory" duration_ms=0 pid=919986gitlab # [ 11.947507] gitaly[919]: time="2026-09-13T12:59:48.830Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-919/git-exec-1201754048.d/git pid=919987gitlab # [ 11.950484] gitaly[919]: time="2026-09-13T12:59:48.835Z" level=info msg="finished detecting git version" duration_ms=5 pid=919988gitlab # [ 11.953020] gitaly[919]: time="2026-09-13T12:59:48.835Z" level=info msg="using Git version" pid=919 version=2.53.g989gitlab # [ 11.955141] gitaly[919]: time="2026-09-13T12:59:48.836Z" level=info msg="clearing disk cache object folder" pid=919 storage=default990gitlab # [ 11.957783] gitaly[919]: time="2026-09-13T12:59:48.836Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3264275945 pid=919 storage=default991gitlab # [ 11.961689] gitaly[919]: time="2026-09-13T12:59:48.836Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3264275945 pid=919 storage=default992gitlab # [ 11.966103] gitaly[919]: time="2026-09-13T12:59:48.836Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=919993gitlab # [ 11.971186] gitaly[919]: time="2026-09-13T12:59:48.836Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=919994gitlab # [ 11.975831] gitaly[919]: time="2026-09-13T12:59:48.836Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3264275945 clear_duration_ms=0 pid=919 storage=default995gitlab # [ 11.980128] gitaly[919]: time="2026-09-13T12:59:48.864Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=919996gitlab # [ 11.983550] gitaly[919]: time="2026-09-13T12:59:48.864Z" level=info msg="counting repositories" pid=919997gitlab # [ 11.985608] gitaly[919]: time="2026-09-13T12:59:48.865Z" level=info msg="starting to count repositories" pid=919 storage_path=/var/gitlab/state/repositories998gitlab # [ 11.988870] gitaly[919]: time="2026-09-13T12:59:48.865Z" level=info msg="finished tempdir cleaner walk" pid=919 storage=default time_ms=0999gitlab # [ 11.991912] gitaly[919]: time="2026-09-13T12:59:48.865Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=919 storage_path=/var/gitlab/state/repositories1000gitlab # [ 11.994772] gitaly[919]: time="2026-09-13T12:59:48.865Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9191001gitlab # [ 11.997016] gitaly[919]: time="2026-09-13T12:59:48.864Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=919 schema=unix1002gitlab # [ 12.000259] gitaly[919]: time="2026-09-13T12:59:48.865Z" level=info msg="listening at address" address=/run/gitaly/gitaly-919/sock.d/intern pid=919 schema=unix1003gitlab # [ 12.003590] gitaly[919]: time="2026-09-13T12:59:48.866Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9191004gitlab # [ 12.006572] gitaly[919]: time="2026-09-13T12:59:48.866Z" level=info msg="Started Gitaly" duration_ms=263 pid=9191005gitlab # [ 12.009196] gitaly[919]: time="2026-09-13T12:59:48.866Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9191006gitlab # [ 12.011944] gitaly[919]: time="2026-09-13T12:59:48.867Z" level=info msg="maintenance: daily scheduled" pid=919 scheduled="2026-09-14 12:00:00 +0000 UTC"1007gitlab # [ 12.309801] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[930]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1008gitlab # [ 12.319839] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[930]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1009gitlab # [ 12.329449] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[930]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1010gitlab # [ 13.519338] dhcpcd[750]: eth0: leased 10.0.2.15 for 86400 seconds1011gitlab # [ 13.519662] dhcpcd[750]: eth0: adding route to 10.0.2.0/241012gitlab # [ 13.519795] dhcpcd[750]: eth0: adding default route via 10.0.2.21013gitlab # [ 13.683601] systemd[1]: Started DHCP Client.1014gitlab # [ 13.685985] systemd[1]: Reached target Network is Online.1015gitlab # [ 15.334878] gitaly[919]: time="2026-09-13T12:59:52.222Z" level=info msg="License database preloaded" duration_ms=3619 pid=9191016gitlab # [ 24.008937] systemd[1]: Starting Logrotate Service...1017gitlab # [ 24.145039] systemd[1]: logrotate.service: Deactivated successfully.1018gitlab # [ 24.147821] systemd[1]: Finished Logrotate Service.1019gitlab # [ 58.007377] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[930]: W, [2026-09-13T13:00:34.894868 #930] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1020gitlab # [ 59.082157] gitlab-workhorse[819]: time="2026-09-13T13:00:35Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1021gitlab # [ 61.118119] postgres[1441]: [1441] ERROR: relation "feature_gates" does not exist at character 5231022gitlab # [ 61.118350] postgres[1441]: [1441] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1023gitlab # [ 61.118430] postgres[1441]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1024gitlab # [ 61.118483] postgres[1441]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1025gitlab # [ 61.118533] postgres[1441]: attidentity AS identity,1026gitlab # [ 61.118579] postgres[1441]: attgenerated as attgenerated1027gitlab # [ 61.118626] postgres[1441]: FROM pg_attribute a1028gitlab # [ 61.118672] postgres[1441]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1029gitlab # [ 61.118713] postgres[1441]: LEFT JOIN pg_type t ON a.atttypid = t.oid1030gitlab # [ 61.118759] postgres[1441]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1031gitlab # [ 61.118805] postgres[1441]: WHERE a.attrelid = '"feature_gates"'::regclass1032gitlab # [ 61.118859] postgres[1441]: AND a.attnum > 0 AND NOT a.attisdropped1033gitlab # [ 61.118899] postgres[1441]: ORDER BY a.attnum1034gitlab # [ 61.118944] postgres[1441]:1035gitlab # [ 67.120631] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1501]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1036gitlab # [ 67.121053] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1501]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1037gitlab # [ 69.091617] gitlab-workhorse[819]: time="2026-09-13T13:00:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 79.102883] gitlab-workhorse[819]: time="2026-09-13T13:00:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1039gitlab # [ 89.110079] gitlab-workhorse[819]: time="2026-09-13T13:01:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1040gitlab # [ 92.187410] postgres[1604]: [1604] ERROR: relation "table_without_models" does not exist1041gitlab # [ 92.187789] postgres[1604]: [1604] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1042gitlab # [ 99.114530] gitlab-workhorse[819]: time="2026-09-13T13:01:16Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1043gitlab # [ 109.122156] gitlab-workhorse[819]: time="2026-09-13T13:01:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1044gitlab # [ 119.130527] gitlab-workhorse[819]: time="2026-09-13T13:01:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1045gitlab # [ 123.364099] systemd[1]: Finished gitlab-db-config.service.1046gitlab # [ 123.369976] systemd[1]: Started gitlab-sidekiq.service.1047gitlab # [ 123.375466] systemd[1]: Starting gitlab.service...1048gitlab # [ 123.970179] bundle[1979]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1049gitlab # [ 123.976511] bundle[1979]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1050gitlab # [ 123.982353] bundle[1979]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1051gitlab # [ 128.574102] sidekiq-cluster[1978]: {"severity":"INFO","time":"2026-09-13T13:01:45.431Z","message":"Starting cluster with 1 processes"}1052gitlab # [ 128.700642] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.559Z","pid":1979,"message":"* Enabling systemd notification integration"}1053gitlab # [ 128.705802] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"Puma starting in cluster mode..."}1054gitlab # [ 128.710852] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1055gitlab # [ 128.716250] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1056gitlab # [ 128.722359] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Min threads: 0"}1057gitlab # [ 128.726623] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Max threads: 4"}1058gitlab # [ 128.730802] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Environment: production"}1059gitlab # [ 128.735342] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Master PID: 1979"}1060gitlab # [ 128.739363] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Workers: 2"}1061gitlab # [ 128.743351] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1062gitlab # [ 128.749531] bundle[1979]: {"timestamp":"2026-09-13T13:01:45.588Z","pid":1979,"message":"* Preloading application"}1063gitlab # [ 129.135949] gitlab-workhorse[819]: time="2026-09-13T13:01:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1064gitlab # [ 139.145920] gitlab-workhorse[819]: time="2026-09-13T13:01:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1065gitlab # [ 149.151344] gitlab-workhorse[819]: time="2026-09-13T13:02:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1066gitlab # [ 159.152573] gitlab-workhorse[819]: time="2026-09-13T13:02:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1067gitlab # [ 168.077387] bundle[1979]: W, [2026-09-13T13:02:24.964458 #1979] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1068gitlab # [ 169.155548] gitlab-workhorse[819]: time="2026-09-13T13:02:26Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1069gitlab # [ 173.507867] sidekiq-cluster[2027]: 2026-09-13T13:02:30.395Z pid=2027 tid=wn7 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2027, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431353856}1070gitlab # [ 173.824066] sidekiq-cluster[2027]: W, [2026-09-13T13:02:30.712750 #2027] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1071gitlab # [ 173.891707] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:30.780Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1072gitlab # [ 177.728360] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.615Z","message":"GitLab reliable fetch activated!"}1073gitlab # [ 177.728726] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.616Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1074gitlab # [ 177.763283] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.652Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1075gitlab # [ 177.770461] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.655Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1076gitlab # [ 177.777550] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.658Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1077gitlab # [ 177.784118] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.661Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1078gitlab # [ 177.791385] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.663Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1079gitlab # [ 177.797934] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.666Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1080gitlab # [ 177.803834] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.668Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1081gitlab # [ 177.808715] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.670Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1082gitlab # [ 177.813099] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.672Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1083gitlab # [ 177.817520] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.674Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1084gitlab # [ 177.822205] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.676Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1085gitlab # [ 177.826419] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.679Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1086gitlab # [ 177.830067] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.681Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1087gitlab # [ 177.833750] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.683Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1088gitlab # [ 177.837198] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.685Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1089gitlab # [ 177.840469] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.687Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1090gitlab # [ 177.843344] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.690Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1091gitlab # [ 177.846244] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.693Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1092gitlab # [ 177.849254] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.696Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1093gitlab # [ 177.852093] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.699Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1094gitlab # [ 177.855050] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.702Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1095gitlab # [ 177.858342] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.704Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1096gitlab # [ 177.861131] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.706Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1097gitlab # [ 177.863919] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.708Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1098gitlab # [ 177.866967] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.710Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1099gitlab # [ 177.869786] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.713Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1100gitlab # [ 177.872412] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.716Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1101gitlab # [ 177.875254] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.719Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1102gitlab # [ 177.878100] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.722Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1103gitlab # [ 177.880992] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.725Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1104gitlab # [ 177.883587] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.728Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1105gitlab # [ 177.886676] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.730Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1106gitlab # [ 177.889571] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.733Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1107gitlab # [ 177.892394] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.736Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1108gitlab # [ 177.895096] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.738Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1109gitlab # [ 177.897909] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.745Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1110gitlab # [ 177.900820] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.753Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1111gitlab # [ 177.903423] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.760Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1112gitlab # [ 177.906198] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.765Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1113gitlab # [ 177.908894] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.771Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1114gitlab # [ 177.911668] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.776Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1115gitlab # [ 177.914456] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.779Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1116gitlab # [ 177.917170] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.783Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1117gitlab # [ 177.919603] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.787Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1118gitlab # [ 177.922272] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.791Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1119gitlab # [ 177.924898] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.794Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1120gitlab # [ 177.927247] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.797Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1121gitlab # [ 177.929773] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.800Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1122gitlab # [ 177.931948] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.803Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1123gitlab # [ 177.934341] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.806Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1124gitlab # [ 177.936603] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.809Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1125gitlab # [ 177.938693] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.812Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1126gitlab # [ 177.940909] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.828Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1127gitlab # [ 177.943275] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.831Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1128gitlab # [ 177.945558] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.834Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1129gitlab # [ 177.947664] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.836Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1130gitlab # [ 177.949925] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.838Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1131gitlab # [ 177.952492] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.841Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1132gitlab # [ 177.955312] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.843Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1133gitlab # [ 177.957618] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.845Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1134gitlab # [ 177.959589] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.847Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1135gitlab # [ 177.961694] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.849Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1136gitlab # [ 177.963772] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.851Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1137gitlab # [ 177.966077] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.854Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1138gitlab # [ 177.968295] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.856Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1139gitlab # [ 177.970444] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.858Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1140gitlab # [ 177.972482] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.860Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1141gitlab # [ 177.974562] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.862Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1142gitlab # [ 177.976796] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.863Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1143gitlab # [ 177.979017] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.865Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1144gitlab # [ 177.981237] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.866Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1145gitlab # [ 177.983388] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.868Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1146gitlab # [ 177.985679] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.870Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1147gitlab # [ 177.987955] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.871Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1148gitlab # [ 177.990396] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.873Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1149gitlab # [ 177.992726] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.874Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1150gitlab # [ 177.994871] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.876Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1151gitlab # [ 177.997380] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.878Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1152gitlab # [ 177.999532] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.879Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1153gitlab # [ 178.001743] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.881Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1154gitlab # [ 178.004029] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.883Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1155gitlab # [ 178.006377] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.884Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1156gitlab # [ 178.008653] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.887Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1157gitlab # [ 178.010873] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.889Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1158gitlab # [ 178.013290] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.890Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1159gitlab # [ 178.015429] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.892Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1160gitlab # [ 178.017854] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.894Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1161gitlab # [ 178.020059] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.896Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1162gitlab # [ 178.022418] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.898Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1163gitlab # [ 178.024692] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.900Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1164gitlab # [ 178.027016] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.901Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1165gitlab # [ 178.029278] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.903Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1166gitlab # [ 178.031431] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.905Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1167gitlab # [ 178.033941] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.906Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1168gitlab # [ 178.035909] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.909Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1169gitlab # [ 178.038428] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.911Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1170gitlab # [ 178.040719] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.913Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1171gitlab # [ 178.042908] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.915Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1172gitlab # [ 178.045164] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.925Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1173gitlab # [ 178.047315] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.927Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1174gitlab # [ 178.049549] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.929Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1175gitlab # [ 178.051765] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.932Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1176gitlab # [ 178.053949] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.935Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1177gitlab # [ 178.056120] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.937Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1178gitlab # [ 178.058452] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.940Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1179gitlab # [ 178.060876] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.942Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1180gitlab # [ 178.062919] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.944Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1181gitlab # [ 178.065174] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.947Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1182gitlab # [ 178.105946] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:02:34.994Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1183gitlab # [ 179.156990] gitlab-workhorse[819]: time="2026-09-13T13:02:36Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1184gitlab # [ 189.158199] gitlab-workhorse[819]: time="2026-09-13T13:02:46Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1185gitlab # [ 199.166245] gitlab-workhorse[819]: time="2026-09-13T13:02:56Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1186gitlab # [ 209.173418] gitlab-workhorse[819]: time="2026-09-13T13:03:06Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1187gitlab # [ 218.297100] postgres[2784]: [2784] ERROR: relation "table_without_models" does not exist1188gitlab # [ 218.297494] postgres[2784]: [2784] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1189gitlab # [ 219.001773] postgres[2795]: [2795] ERROR: relation "table_without_models" does not exist1190gitlab # [ 219.002098] postgres[2795]: [2795] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1191gitlab # [ 219.183423] gitlab-workhorse[819]: time="2026-09-13T13:03:16Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1192gitlab # [ 223.222891] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:20.111Z","message":"Booted Rails 7.2.3.1 application in production environment"}1193gitlab # [ 223.226556] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:20.115Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1194gitlab # [ 223.228679] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:20.117Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1195gitlab # [ 223.230412] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:20.117Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1196gitlab # [ 223.232570] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:20.121Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1197gitlab # [ 224.573026] bundle[1979]: {"timestamp":"2026-09-13T13:03:21.459Z","pid":1979,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1198gitlab # [ 224.579005] bundle[1979]: {"timestamp":"2026-09-13T13:03:21.459Z","pid":1979,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1199gitlab # [ 224.584299] bundle[1979]: {"timestamp":"2026-09-13T13:03:21.460Z","pid":1979,"message":"! #\u003cThread:0x0000fdf6275e96f8 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1200gitlab # [ 224.599837] bundle[1979]: {"timestamp":"2026-09-13T13:03:21.460Z","pid":1979,"message":"! #\u003cThread:0x0000fdf62661ae70 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1201gitlab # [ 224.615184] bundle[1979]: {"timestamp":"2026-09-13T13:03:21.460Z","pid":1979,"message":"Use Ctrl-C to stop"}1202gitlab # [ 224.618663] bundle[1979]: {"timestamp":"2026-09-13T13:03:21.461Z","pid":1979,"message":"Sending status to systemd every 1.0 sec"}1203gitlab # [ 225.315327] bundle[1979]: {"timestamp":"2026-09-13T13:03:22.204Z","pid":1979,"message":"- Worker 0 (PID: 2888) booted in 0.69s, phase: 0"}1204gitlab # [ 225.383390] bundle[1979]: {"timestamp":"2026-09-13T13:03:22.272Z","pid":1979,"message":"- Worker 1 (PID: 2892) booted in 0.73s, phase: 0"}1205gitlab # [ 225.392864] systemd[1]: Started gitlab.service.1206gitlab # [ 225.394800] systemd[1]: Reached target Common target for all GitLab services..1207gitlab # [ 225.401154] systemd[1]: Starting setup-gitlab.service...1208gitlab # [ 225.446075] setup-gitlab-start[2925]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1209gitlab # [ 225.948697] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:22.836Z","message":"Cleaning working queues"}1210gitlab # [ 225.953732] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:22.842Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c8ff5373b01e210de33d3b3f","created_at":"2026-09-13T13:01:39.711Z","trace_propagation_headers":{"sentry-trace":"529312341f05479cad089accfec00bea-b1e72b844ee04150","baggage":"sentry-trace_id=529312341f05479cad089accfec00bea,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"f3bfb5cf0113c5af1f33956e0d0b9346","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T13:01:39.723Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c8ff5373b01e210de33d3b3f: start","job_status":"start","queue_duration_s":103.11872,"scheduling_latency_s":103.118782}1211gitlab # [ 226.273984] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:23.161Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c8ff5373b01e210de33d3b3f","created_at":"2026-09-13T13:01:39.711Z","trace_propagation_headers":{"sentry-trace":"529312341f05479cad089accfec00bea-b1e72b844ee04150","baggage":"sentry-trace_id=529312341f05479cad089accfec00bea,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"f3bfb5cf0113c5af1f33956e0d0b9346","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T13:01:39.723Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c8ff5373b01e210de33d3b3f: done: 0.319846 sec","job_status":"done","queue_duration_s":103.11872,"scheduling_latency_s":103.118782,"redis_calls":24,"redis_duration_s":0.008902,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.004214,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002882,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001806,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.171804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.20925434,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.319846,"completed_at":"2026-09-13T13:03:23.161Z","load_balancing_strategy":"primary","db_duration_s":0.103393,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1212gitlab # [ 229.047272] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:03:25 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 35471213gitlab # [ 229.053697] setup-gitlab-start[2925]: + TOKEN=nixbot-test-token-of-20-chars1214gitlab # [ 229.055852] setup-gitlab-start[2925]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1215gitlab # [ 229.071555] runuser[2958]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1216gitlab # [ 229.646133] setup-gitlab-start[2959]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1217gitlab # [ 229.652710] setup-gitlab-start[2959]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1218gitlab # [ 229.658940] setup-gitlab-start[2959]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1219gitlab # [ 230.483263] setup-gitlab-start[2959]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1220gitlab # [ 230.483832] setup-gitlab-start[2959]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1221gitlab # [ 230.485958] setup-gitlab-start[2959]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1222gitlab # [ 238.963309] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:35.850Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0267f4450bd918077a22deb6","created_at":"2026-09-13T13:03:35.773Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37a6c1353513c274d47862efeea3929f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:35.846Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0267f4450bd918077a22deb6: start","job_status":"start","queue_duration_s":0.00368,"scheduling_latency_s":0.003746}1223gitlab # [ 239.965067] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:36.846Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0267f4450bd918077a22deb6","created_at":"2026-09-13T13:03:35.773Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37a6c1353513c274d47862efeea3929f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:35.846Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0267f4450bd918077a22deb6: done: 0.995735 sec","job_status":"done","queue_duration_s":0.00368,"scheduling_latency_s":0.003746,"redis_calls":12,"redis_duration_s":0.033848,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002569,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.017932,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.013347,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.151,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.151,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.309495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00554418,"exclusive_lock_hold_duration_s":0.53793238,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.995735,"completed_at":"2026-09-13T13:03:36.845Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.219463,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1224gitlab # [ 240.060274] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:36.859Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b00a64973f0da19f439eff30","created_at":"2026-09-13T13:03:35.869Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e0df0e6543e7335e8a2b20fc6e2c2e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:35.914Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b00a64973f0da19f439eff30: start","job_status":"start","queue_duration_s":0.944192,"scheduling_latency_s":0.944329}1225gitlab # [ 240.520205] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:37.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"b00a64973f0da19f439eff30","created_at":"2026-09-13T13:03:35.869Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e0df0e6543e7335e8a2b20fc6e2c2e5","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:35.914Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-b00a64973f0da19f439eff30: done: 0.545947 sec","job_status":"done","queue_duration_s":0.944192,"scheduling_latency_s":0.944329,"redis_calls":12,"redis_duration_s":0.052781,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.010416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.02097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.021395,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.115,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.115,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.01083894,"exclusive_lock_hold_duration_s":0.10701664,"duration_s":0.545947,"completed_at":"2026-09-13T13:03:37.404Z","load_balancing_strategy":"primary","db_duration_s":0.139093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1226gitlab # [ 240.625688] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:37.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"cf570a3e7204e77a2704e3d2","created_at":"2026-09-13T13:03:35.961Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a60e417947034487c05124174fd35e00","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:35.978Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-cf570a3e7204e77a2704e3d2: start","job_status":"start","queue_duration_s":1.523513,"scheduling_latency_s":1.523738}1227gitlab # [ 241.106498] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:37.988Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"cf570a3e7204e77a2704e3d2","created_at":"2026-09-13T13:03:35.961Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a60e417947034487c05124174fd35e00","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:35.978Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-cf570a3e7204e77a2704e3d2: done: 0.486362 sec","job_status":"done","queue_duration_s":1.523513,"scheduling_latency_s":1.523738,"redis_calls":13,"redis_duration_s":0.085511,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007138,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.044991,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.033382,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.092,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.092,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.181993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.08853896,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.486362,"completed_at":"2026-09-13T13:03:37.988Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.091465,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1228gitlab # [ 241.192797] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:38.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8ff983c9aa408303020bb493","created_at":"2026-09-13T13:03:36.006Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.028Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8ff983c9aa408303020bb493: start","job_status":"start","queue_duration_s":2.050843,"scheduling_latency_s":2.050935}1229gitlab # [ 242.305058] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:39.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"8ff983c9aa408303020bb493","created_at":"2026-09-13T13:03:36.006Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.028Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-8ff983c9aa408303020bb493: done: 1.111556 sec","job_status":"done","queue_duration_s":2.050843,"scheduling_latency_s":2.050935,"redis_calls":71,"redis_duration_s":0.17977300000000002,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.008781,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00466,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.159641,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006691,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.573233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00482332,"duration_s":1.111556,"completed_at":"2026-09-13T13:03:39.190Z","load_balancing_strategy":"primary","db_duration_s":0.009941,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1230gitlab # [ 242.399555] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:39.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e821c72a2a5268e61360eaab","created_at":"2026-09-13T13:03:36.045Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d1e68f92f4b32bc711cef4cfec41c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.066Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e821c72a2a5268e61360eaab: start","job_status":"start","queue_duration_s":3.142686,"scheduling_latency_s":3.142752}1231gitlab # [ 242.480273] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:39.367Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e821c72a2a5268e61360eaab","created_at":"2026-09-13T13:03:36.045Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77d1e68f92f4b32bc711cef4cfec41c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.066Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e821c72a2a5268e61360eaab: done: 0.158624 sec","job_status":"done","queue_duration_s":3.142686,"scheduling_latency_s":3.142752,"redis_calls":12,"redis_duration_s":0.042299,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.014464,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.024607,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003228,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00619842,"duration_s":0.158624,"completed_at":"2026-09-13T13:03:39.367Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018899,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1232gitlab # [ 242.580808] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:39.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"11a2bd7ba4363ab5f7f09654","created_at":"2026-09-13T13:03:36.119Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"813994a75f6606f3b5ed84129ddc5e98","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.147Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-11a2bd7ba4363ab5f7f09654: start","job_status":"start","queue_duration_s":3.241227,"scheduling_latency_s":3.241351}1233gitlab # [ 245.285455] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"11a2bd7ba4363ab5f7f09654","created_at":"2026-09-13T13:03:36.119Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"813994a75f6606f3b5ed84129ddc5e98","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.147Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-11a2bd7ba4363ab5f7f09654: done: 2.782191 sec","job_status":"done","queue_duration_s":3.241227,"scheduling_latency_s":3.241351,"redis_calls":565,"redis_duration_s":1.780679,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.008059,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.019435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":1.753185,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.118899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00199238,"duration_s":2.782191,"completed_at":"2026-09-13T13:03:42.171Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004658,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1234gitlab # [ 245.292246] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.178Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"7b9ecf25bf163c476474f98e","created_at":"2026-09-13T13:03:36.304Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d881b6adbefee9e05aaf479d2041d92d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.337Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-7b9ecf25bf163c476474f98e: start","job_status":"start","queue_duration_s":5.840861,"scheduling_latency_s":5.840927}1235gitlab # [ 245.633974] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PipelineScheduleWorker","jid":"7b9ecf25bf163c476474f98e","created_at":"2026-09-13T13:03:36.304Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d881b6adbefee9e05aaf479d2041d92d","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f6baa6e6d21754c4b13358fe49f8c31a08729a9dd276fb680a810caf1bc7fab5","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.337Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineScheduleWorker JID-7b9ecf25bf163c476474f98e: done: 0.343199 sec","job_status":"done","queue_duration_s":5.840861,"scheduling_latency_s":5.840927,"redis_calls":12,"redis_duration_s":0.037057,"redis_read_bytes":11,"redis_write_bytes":1281,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021013,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":518,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.01055,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":476,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.065291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00517782,"exclusive_lock_hold_duration_s":0.11954854,"duration_s":0.343199,"completed_at":"2026-09-13T13:03:42.521Z","load_balancing_strategy":"primary","db_duration_s":0.063907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1236gitlab # [ 245.739151] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d1d4ddedbdd150cc8c581a32","created_at":"2026-09-13T13:03:36.353Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39aa7d8c801f041fbfc91b7a062d44a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.380Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d1d4ddedbdd150cc8c581a32: start","job_status":"start","queue_duration_s":6.146741,"scheduling_latency_s":6.146804}1237gitlab # [ 245.755008] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.598Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1238gitlab # [ 245.758651] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d1d4ddedbdd150cc8c581a32","created_at":"2026-09-13T13:03:36.353Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"39aa7d8c801f041fbfc91b7a062d44a4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.380Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d1d4ddedbdd150cc8c581a32: done: 0.108208 sec","job_status":"done","queue_duration_s":6.146741,"scheduling_latency_s":6.146804,"redis_calls":10,"redis_duration_s":0.006994,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002682,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041532,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00082882,"duration_s":0.108208,"completed_at":"2026-09-13T13:03:42.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1239gitlab # [ 245.800552] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1e410d7672d3eb68a299b93a","created_at":"2026-09-13T13:03:36.399Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9990aa5d8b3fa3bc41295e400dbd0775","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T13:03:36.421Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1e410d7672d3eb68a299b93a: start","job_status":"start","queue_duration_s":6.222501,"scheduling_latency_s":6.222567}1240gitlab # [ 245.813923] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.699Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1241gitlab # [ 245.847034] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1e410d7672d3eb68a299b93a","created_at":"2026-09-13T13:03:36.399Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9990aa5d8b3fa3bc41295e400dbd0775","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T13:03:36.421Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1e410d7672d3eb68a299b93a: done: 0.090809 sec","job_status":"done","queue_duration_s":6.222501,"scheduling_latency_s":6.222567,"redis_calls":10,"redis_duration_s":0.004994,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001111,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00137048,"duration_s":0.090809,"completed_at":"2026-09-13T13:03:42.734Z","load_balancing_strategy":"primary","db_duration_s":0.001978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1242gitlab # [ 245.893796] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b3c8badb54a0b0154960400f","created_at":"2026-09-13T13:03:36.452Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.473Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b3c8badb54a0b0154960400f: start","job_status":"start","queue_duration_s":6.26753,"scheduling_latency_s":6.267598}1243gitlab # [ 246.101314] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:42.988Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b3c8badb54a0b0154960400f","created_at":"2026-09-13T13:03:36.452Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.473Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b3c8badb54a0b0154960400f: done: 0.248091 sec","job_status":"done","queue_duration_s":6.26753,"scheduling_latency_s":6.267598,"redis_calls":19,"redis_duration_s":0.077954,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.017019,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.00923,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.041046,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.010659,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0041656,"duration_s":0.248091,"completed_at":"2026-09-13T13:03:42.988Z","load_balancing_strategy":"primary","db_duration_s":0.019794,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1244gitlab # [ 246.240109] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.128Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fcd1ea631d92c05548468d1a","created_at":"2026-09-13T13:03:36.489Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"192a6d4db2138c89d45305183476ce4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T13:03:36.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fcd1ea631d92c05548468d1a: start","job_status":"start","queue_duration_s":6.591503,"scheduling_latency_s":6.591566}1245gitlab # [ 246.485378] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fcd1ea631d92c05548468d1a","created_at":"2026-09-13T13:03:36.489Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"192a6d4db2138c89d45305183476ce4a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T13:03:36.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fcd1ea631d92c05548468d1a: done: 0.242582 sec","job_status":"done","queue_duration_s":6.591503,"scheduling_latency_s":6.591566,"redis_calls":10,"redis_duration_s":0.026671,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005117,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016603,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004951,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00485134,"duration_s":0.242582,"completed_at":"2026-09-13T13:03:43.370Z","load_balancing_strategy":"primary","db_duration_s":0.019295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1246gitlab # [ 246.563645] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.390Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2ce85489de87a4e9add90c3a","created_at":"2026-09-13T13:03:36.575Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b24c55d32fe721bb04f37e41f7779c2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.605Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2ce85489de87a4e9add90c3a: start","job_status":"start","queue_duration_s":6.784664,"scheduling_latency_s":6.784732}1247gitlab # [ 246.710089] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.597Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2ce85489de87a4e9add90c3a","created_at":"2026-09-13T13:03:36.575Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b24c55d32fe721bb04f37e41f7779c2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.605Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2ce85489de87a4e9add90c3a: done: 0.207211 sec","job_status":"done","queue_duration_s":6.784664,"scheduling_latency_s":6.784732,"redis_calls":16,"redis_duration_s":0.051671,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.022975,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.020052,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.008644,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.035,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00216394,"duration_s":0.207211,"completed_at":"2026-09-13T13:03:43.597Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1248gitlab # [ 246.804135] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.604Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2f04a89b508ccd5929b3a4cf","created_at":"2026-09-13T13:03:36.651Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7cb2b3b337a7a514da738bd2d098172","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T13:03:36.703Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2f04a89b508ccd5929b3a4cf: start","job_status":"start","queue_duration_s":6.901302,"scheduling_latency_s":6.901361}1249gitlab # [ 246.820449] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.672Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1250gitlab # [ 246.835064] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"2f04a89b508ccd5929b3a4cf","created_at":"2026-09-13T13:03:36.651Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c7cb2b3b337a7a514da738bd2d098172","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T13:03:36.703Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-2f04a89b508ccd5929b3a4cf: done: 0.11824 sec","job_status":"done","queue_duration_s":6.901302,"scheduling_latency_s":6.901361,"redis_calls":10,"redis_duration_s":0.019435,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005631,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009487,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004317,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00486882,"duration_s":0.11824,"completed_at":"2026-09-13T13:03:43.722Z","load_balancing_strategy":"primary","db_duration_s":0.011989,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1251gitlab # [ 246.892057] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.740Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"547cae5b0ce0e5b4b2042c4d","created_at":"2026-09-13T13:03:36.814Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a7f4a8e00829d1f6d12d53b5325a72b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.879Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-547cae5b0ce0e5b4b2042c4d: start","job_status":"start","queue_duration_s":6.860729,"scheduling_latency_s":6.860793}1252gitlab # [ 247.033742] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:43.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"547cae5b0ce0e5b4b2042c4d","created_at":"2026-09-13T13:03:36.814Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a7f4a8e00829d1f6d12d53b5325a72b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:36.879Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-547cae5b0ce0e5b4b2042c4d: done: 0.180614 sec","job_status":"done","queue_duration_s":6.860729,"scheduling_latency_s":6.860793,"redis_calls":10,"redis_duration_s":0.094943,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.07162,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.02094,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037221,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00241142,"duration_s":0.180614,"completed_at":"2026-09-13T13:03:43.920Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003791,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1253gitlab # [ 247.176301] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.064Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"858e56ed44dd2e29196ab291","created_at":"2026-09-13T13:03:36.982Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e04de9ad4e83f4d84dc3074917670e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.039Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-858e56ed44dd2e29196ab291: start","job_status":"start","queue_duration_s":7.024994,"scheduling_latency_s":7.025054}1254gitlab # [ 247.306691] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.195Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1255gitlab # [ 247.379251] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.266Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"858e56ed44dd2e29196ab291","created_at":"2026-09-13T13:03:36.982Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e04de9ad4e83f4d84dc3074917670e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.039Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-858e56ed44dd2e29196ab291: done: 0.202642 sec","job_status":"done","queue_duration_s":7.024994,"scheduling_latency_s":7.025054,"redis_calls":10,"redis_duration_s":0.018951,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003748,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004139,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00206798,"duration_s":0.202642,"completed_at":"2026-09-13T13:03:44.266Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1256gitlab # [ 247.521504] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1846fffc497c9db057b490b6","created_at":"2026-09-13T13:03:37.144Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9190d61fc90d121d2ce67b6c151713c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.211Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1846fffc497c9db057b490b6: start","job_status":"start","queue_duration_s":7.19689,"scheduling_latency_s":7.196951}1257gitlab # [ 247.650674] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"1846fffc497c9db057b490b6","created_at":"2026-09-13T13:03:37.144Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e9190d61fc90d121d2ce67b6c151713c","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.211Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-1846fffc497c9db057b490b6: done: 0.129983 sec","job_status":"done","queue_duration_s":7.19689,"scheduling_latency_s":7.196951,"redis_calls":10,"redis_duration_s":0.004307,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0011,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00071374,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.129983,"completed_at":"2026-09-13T13:03:44.538Z","load_balancing_strategy":"primary","db_duration_s":0.018388,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 247.751173] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.541Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c4929e7571d55afc7c4e2440","created_at":"2026-09-13T13:03:37.297Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af402d17d06ed07570ed19eb0747dc6e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.418Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c4929e7571d55afc7c4e2440: start","job_status":"start","queue_duration_s":7.122822,"scheduling_latency_s":7.122871}1259gitlab # [ 247.774382] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c4929e7571d55afc7c4e2440","created_at":"2026-09-13T13:03:37.297Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"af402d17d06ed07570ed19eb0747dc6e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.418Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c4929e7571d55afc7c4e2440: done: 0.119941 sec","job_status":"done","queue_duration_s":7.122822,"scheduling_latency_s":7.122871,"redis_calls":14,"redis_duration_s":0.005574,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0016,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001736,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.027,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00115768,"exclusive_lock_hold_duration_s":0.06234566,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.119941,"completed_at":"2026-09-13T13:03:44.661Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034097,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1260gitlab # [ 247.824662] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.665Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d82ba96659867024b6647ca6","created_at":"2026-09-13T13:03:37.490Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"833d23f045c88b1d1ff5b59edf39b9b0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.524Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d82ba96659867024b6647ca6: start","job_status":"start","queue_duration_s":7.140836,"scheduling_latency_s":7.140876}1261gitlab # [ 247.838570] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.714Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"d82ba96659867024b6647ca6","created_at":"2026-09-13T13:03:37.490Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"833d23f045c88b1d1ff5b59edf39b9b0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.524Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-d82ba96659867024b6647ca6: done: 0.049195 sec","job_status":"done","queue_duration_s":7.140836,"scheduling_latency_s":7.140876,"redis_calls":10,"redis_duration_s":0.003281,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000923,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001725,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000633,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053966,"duration_s":0.049195,"completed_at":"2026-09-13T13:03:44.714Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005187,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1262gitlab # [ 247.880034] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5d6f33f68959c3744cf6a0f4","created_at":"2026-09-13T13:03:37.564Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03a67ff54e809c3d437e93727cf4c7e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.587Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5d6f33f68959c3744cf6a0f4: start","job_status":"start","queue_duration_s":7.129728,"scheduling_latency_s":7.129765}1263gitlab # [ 247.894239] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.738Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1264gitlab # [ 247.897410] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.750Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5d6f33f68959c3744cf6a0f4","created_at":"2026-09-13T13:03:37.564Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03a67ff54e809c3d437e93727cf4c7e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.587Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5d6f33f68959c3744cf6a0f4: done: 0.033766 sec","job_status":"done","queue_duration_s":7.129728,"scheduling_latency_s":7.129765,"redis_calls":10,"redis_duration_s":0.002641,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001462,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000604,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042488,"duration_s":0.033766,"completed_at":"2026-09-13T13:03:44.750Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1265gitlab # [ 247.940567] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:44.753Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"19b50d94d10e5b3ae63ca2bb","created_at":"2026-09-13T13:03:37.725Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e09ea428a4f92e6961539c438ea9b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.838Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-19b50d94d10e5b3ae63ca2bb: start","job_status":"start","queue_duration_s":6.914824,"scheduling_latency_s":6.914854}1266gitlab # [ 248.634441] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"19b50d94d10e5b3ae63ca2bb","created_at":"2026-09-13T13:03:37.725Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e09ea428a4f92e6961539c438ea9b6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:37.838Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-19b50d94d10e5b3ae63ca2bb: done: 0.767795 sec","job_status":"done","queue_duration_s":6.914824,"scheduling_latency_s":6.914854,"redis_calls":12,"redis_duration_s":0.003309,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000367,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001596,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001346,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.068,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.068,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.331113,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006293,"exclusive_lock_hold_duration_s":0.73274832,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.767795,"completed_at":"2026-09-13T13:03:45.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.453984,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1267gitlab # [ 248.636595] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.524Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"916e89294df1babf6a6cdeb8","created_at":"2026-09-13T13:03:37.899Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce64425904c9943d5d55956457867d6d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:38.000Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-916e89294df1babf6a6cdeb8: start","job_status":"start","queue_duration_s":7.523584,"scheduling_latency_s":7.523617}1268gitlab # [ 248.769800] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.655Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"916e89294df1babf6a6cdeb8","created_at":"2026-09-13T13:03:37.899Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ce64425904c9943d5d55956457867d6d","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:38.000Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-916e89294df1babf6a6cdeb8: done: 0.131856 sec","job_status":"done","queue_duration_s":7.523584,"scheduling_latency_s":7.523617,"redis_calls":12,"redis_duration_s":0.0045320000000000004,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000759,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001668,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002105,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.083,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.083,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00194104,"exclusive_lock_hold_duration_s":0.007808740000000001,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.131856,"completed_at":"2026-09-13T13:03:45.655Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.082662,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1269gitlab # [ 248.827759] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.661Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b1bcebcb589729e86430ff99","created_at":"2026-09-13T13:03:38.046Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b990dbbe5cfdbe0a306b8d25568cb36f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:38.067Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b1bcebcb589729e86430ff99: start","job_status":"start","queue_duration_s":7.593891,"scheduling_latency_s":7.593938}1270gitlab # [ 248.842270] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.687Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1271gitlab # [ 248.845787] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"b1bcebcb589729e86430ff99","created_at":"2026-09-13T13:03:38.046Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b990dbbe5cfdbe0a306b8d25568cb36f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T13:03:38.067Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-b1bcebcb589729e86430ff99: done: 0.039244 sec","job_status":"done","queue_duration_s":7.593891,"scheduling_latency_s":7.593938,"redis_calls":10,"redis_duration_s":0.002653,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000551,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00057,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044282,"duration_s":0.039244,"completed_at":"2026-09-13T13:03:45.700Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000829,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1272gitlab # [ 248.887642] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.702Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34b00c0245af3df37f18e20c","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34b00c0245af3df37f18e20c: start","job_status":"start","queue_duration_s":6.570117,"scheduling_latency_s":6.570148}1273gitlab # [ 248.900216] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34b00c0245af3df37f18e20c","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34b00c0245af3df37f18e20c: done: 0.038453 sec","job_status":"done","queue_duration_s":6.570117,"scheduling_latency_s":6.570148,"redis_calls":16,"redis_duration_s":0.003502,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038102,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038453,"completed_at":"2026-09-13T13:03:45.740Z","load_balancing_strategy":"primary","db_duration_s":0.000728,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1274gitlab # [ 248.940547] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afe288d8f708061b867f6f0e","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afe288d8f708061b867f6f0e: start","job_status":"start","queue_duration_s":6.610304,"scheduling_latency_s":6.610329}1275gitlab # [ 248.953853] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afe288d8f708061b867f6f0e","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afe288d8f708061b867f6f0e: done: 0.021569 sec","job_status":"done","queue_duration_s":6.610304,"scheduling_latency_s":6.610329,"redis_calls":13,"redis_duration_s":0.002018,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001281,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021569,"completed_at":"2026-09-13T13:03:45.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 248.992831] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb534a3c44035111cd2df92f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb534a3c44035111cd2df92f: start","job_status":"start","queue_duration_s":6.633355,"scheduling_latency_s":6.633376}1277gitlab # [ 249.004827] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb534a3c44035111cd2df92f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb534a3c44035111cd2df92f: done: 0.017687 sec","job_status":"done","queue_duration_s":6.633355,"scheduling_latency_s":6.633376,"redis_calls":13,"redis_duration_s":0.001723,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012255,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017687,"completed_at":"2026-09-13T13:03:45.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1278gitlab # [ 249.043938] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef9f00e713e1a8a89a84a718","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef9f00e713e1a8a89a84a718: start","job_status":"start","queue_duration_s":6.652278,"scheduling_latency_s":6.652296}1279gitlab # [ 249.056455] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef9f00e713e1a8a89a84a718","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef9f00e713e1a8a89a84a718: done: 0.01648 sec","job_status":"done","queue_duration_s":6.652278,"scheduling_latency_s":6.652296,"redis_calls":13,"redis_duration_s":0.0017889999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01648,"completed_at":"2026-09-13T13:03:45.801Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1280gitlab # [ 249.095104] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6e81ab503441c1292bbe9a1","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6e81ab503441c1292bbe9a1: start","job_status":"start","queue_duration_s":6.669906,"scheduling_latency_s":6.669923}1281gitlab # [ 249.107375] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6e81ab503441c1292bbe9a1","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6e81ab503441c1292bbe9a1: done: 0.015686 sec","job_status":"done","queue_duration_s":6.669906,"scheduling_latency_s":6.669923,"redis_calls":13,"redis_duration_s":0.0016150000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000652,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015686,"completed_at":"2026-09-13T13:03:45.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 249.146214] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5979d67e260d7062e65f0f6","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5979d67e260d7062e65f0f6: start","job_status":"start","queue_duration_s":6.687454,"scheduling_latency_s":6.68747}1283gitlab # [ 249.159610] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5979d67e260d7062e65f0f6","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5979d67e260d7062e65f0f6: done: 0.015704 sec","job_status":"done","queue_duration_s":6.687454,"scheduling_latency_s":6.68747,"redis_calls":13,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015704,"completed_at":"2026-09-13T13:03:45.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1284gitlab # [ 249.198572] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5795c5e6cc50c697a1c6a75e","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5795c5e6cc50c697a1c6a75e: start","job_status":"start","queue_duration_s":6.704278,"scheduling_latency_s":6.704295}1285gitlab # [ 249.210984] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5795c5e6cc50c697a1c6a75e","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5795c5e6cc50c697a1c6a75e: done: 0.015383 sec","job_status":"done","queue_duration_s":6.704278,"scheduling_latency_s":6.704295,"redis_calls":13,"redis_duration_s":0.001553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015383,"completed_at":"2026-09-13T13:03:45.851Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 249.251499] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f432f6f6572b812da9e53910","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f432f6f6572b812da9e53910: start","job_status":"start","queue_duration_s":6.720791,"scheduling_latency_s":6.720808}1287gitlab # [ 249.264141] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f432f6f6572b812da9e53910","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f432f6f6572b812da9e53910: done: 0.015983 sec","job_status":"done","queue_duration_s":6.720791,"scheduling_latency_s":6.720808,"redis_calls":13,"redis_duration_s":0.0015569999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015983,"completed_at":"2026-09-13T13:03:45.869Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 249.302741] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67c29910842e339dd2d7f220","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67c29910842e339dd2d7f220: start","job_status":"start","queue_duration_s":6.738036,"scheduling_latency_s":6.738055}1289gitlab # [ 249.315178] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67c29910842e339dd2d7f220","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67c29910842e339dd2d7f220: done: 0.017826 sec","job_status":"done","queue_duration_s":6.738036,"scheduling_latency_s":6.738055,"redis_calls":13,"redis_duration_s":0.0019450000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017826,"completed_at":"2026-09-13T13:03:45.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 249.355026] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1de0bffe18a50fdc8af9a63","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1de0bffe18a50fdc8af9a63: start","job_status":"start","queue_duration_s":6.758346,"scheduling_latency_s":6.758365}1291gitlab # [ 249.367591] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1de0bffe18a50fdc8af9a63","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1de0bffe18a50fdc8af9a63: done: 0.015957 sec","job_status":"done","queue_duration_s":6.758346,"scheduling_latency_s":6.758365,"redis_calls":13,"redis_duration_s":0.0015859999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000986,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015957,"completed_at":"2026-09-13T13:03:45.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 249.406704] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd3629617ced61d8738483c0","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd3629617ced61d8738483c0: start","job_status":"start","queue_duration_s":6.775413,"scheduling_latency_s":6.77543}1293gitlab # [ 249.418988] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fd3629617ced61d8738483c0","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fd3629617ced61d8738483c0: done: 0.01576 sec","job_status":"done","queue_duration_s":6.775413,"scheduling_latency_s":6.77543,"redis_calls":13,"redis_duration_s":0.0016840000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001129,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01576,"completed_at":"2026-09-13T13:03:45.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 249.457856] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16baaced5af428e2da9b064","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16baaced5af428e2da9b064: start","job_status":"start","queue_duration_s":6.792346,"scheduling_latency_s":6.792363}1295gitlab # [ 249.470245] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a16baaced5af428e2da9b064","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a16baaced5af428e2da9b064: done: 0.016394 sec","job_status":"done","queue_duration_s":6.792346,"scheduling_latency_s":6.792363,"redis_calls":13,"redis_duration_s":0.001515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016394,"completed_at":"2026-09-13T13:03:45.941Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 249.509703] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66c303cf20d2f72cf296c7d0","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66c303cf20d2f72cf296c7d0: start","job_status":"start","queue_duration_s":6.809898,"scheduling_latency_s":6.809915}1297gitlab # [ 249.521946] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66c303cf20d2f72cf296c7d0","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66c303cf20d2f72cf296c7d0: done: 0.015137 sec","job_status":"done","queue_duration_s":6.809898,"scheduling_latency_s":6.809915,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015137,"completed_at":"2026-09-13T13:03:45.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 249.561396] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"622ef3995531575fdb82be0f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-622ef3995531575fdb82be0f: start","job_status":"start","queue_duration_s":6.826181,"scheduling_latency_s":6.826198}1299gitlab # [ 249.573519] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"622ef3995531575fdb82be0f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-622ef3995531575fdb82be0f: done: 0.015482 sec","job_status":"done","queue_duration_s":6.826181,"scheduling_latency_s":6.826198,"redis_calls":13,"redis_duration_s":0.0015400000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015482,"completed_at":"2026-09-13T13:03:45.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 249.612883] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4720621a4fff2089b86e261b","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4720621a4fff2089b86e261b: start","job_status":"start","queue_duration_s":6.84275,"scheduling_latency_s":6.842767}1301gitlab # [ 249.625171] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4720621a4fff2089b86e261b","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4720621a4fff2089b86e261b: done: 0.015194 sec","job_status":"done","queue_duration_s":6.84275,"scheduling_latency_s":6.842767,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015194,"completed_at":"2026-09-13T13:03:45.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 249.664417] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:45.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62235cad882387228df65b81","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62235cad882387228df65b81: start","job_status":"start","queue_duration_s":6.859339,"scheduling_latency_s":6.859357}1303gitlab # [ 249.676756] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62235cad882387228df65b81","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62235cad882387228df65b81: done: 0.015205 sec","job_status":"done","queue_duration_s":6.859339,"scheduling_latency_s":6.859357,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015205,"completed_at":"2026-09-13T13:03:46.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 249.715362] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3b8185fd33c3b3dbb7ffa99","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3b8185fd33c3b3dbb7ffa99: start","job_status":"start","queue_duration_s":6.875646,"scheduling_latency_s":6.875663}1305gitlab # [ 249.727821] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.023Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3b8185fd33c3b3dbb7ffa99","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3b8185fd33c3b3dbb7ffa99: done: 0.015991 sec","job_status":"done","queue_duration_s":6.875646,"scheduling_latency_s":6.875663,"redis_calls":13,"redis_duration_s":0.0015999999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015991,"completed_at":"2026-09-13T13:03:46.023Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 249.767874] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d2296ac57fbd677a34ef48f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d2296ac57fbd677a34ef48f: start","job_status":"start","queue_duration_s":6.892761,"scheduling_latency_s":6.892777}1307gitlab # [ 249.780437] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d2296ac57fbd677a34ef48f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d2296ac57fbd677a34ef48f: done: 0.015158 sec","job_status":"done","queue_duration_s":6.892761,"scheduling_latency_s":6.892777,"redis_calls":13,"redis_duration_s":0.002046,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001524,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015158,"completed_at":"2026-09-13T13:03:46.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 249.819643] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88749e24b4dd17ff38ee5cbf","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88749e24b4dd17ff38ee5cbf: start","job_status":"start","queue_duration_s":6.909062,"scheduling_latency_s":6.909079}1309gitlab # [ 249.832284] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88749e24b4dd17ff38ee5cbf","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88749e24b4dd17ff38ee5cbf: done: 0.014989 sec","job_status":"done","queue_duration_s":6.909062,"scheduling_latency_s":6.909079,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014989,"completed_at":"2026-09-13T13:03:46.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 249.871211] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f732bf17c990278fee5293bc","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f732bf17c990278fee5293bc: start","job_status":"start","queue_duration_s":6.925191,"scheduling_latency_s":6.925209}1311gitlab # [ 249.883436] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f732bf17c990278fee5293bc","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f732bf17c990278fee5293bc: done: 0.015119 sec","job_status":"done","queue_duration_s":6.925191,"scheduling_latency_s":6.925209,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015119,"completed_at":"2026-09-13T13:03:46.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 249.922489] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e39d92f08a43759f6e184e4","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e39d92f08a43759f6e184e4: start","job_status":"start","queue_duration_s":6.942075,"scheduling_latency_s":6.942093}1313gitlab # [ 249.934865] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e39d92f08a43759f6e184e4","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e39d92f08a43759f6e184e4: done: 0.015077 sec","job_status":"done","queue_duration_s":6.942075,"scheduling_latency_s":6.942093,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000567,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015077,"completed_at":"2026-09-13T13:03:46.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 249.974548] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c442c9378366fcdbb600d564","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c442c9378366fcdbb600d564: start","job_status":"start","queue_duration_s":6.95829,"scheduling_latency_s":6.958306}1315gitlab # [ 249.987130] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c442c9378366fcdbb600d564","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c442c9378366fcdbb600d564: done: 0.015199 sec","job_status":"done","queue_duration_s":6.95829,"scheduling_latency_s":6.958306,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015199,"completed_at":"2026-09-13T13:03:46.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 250.026833] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bae59a20b49d2270aa516e5","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bae59a20b49d2270aa516e5: start","job_status":"start","queue_duration_s":6.974591,"scheduling_latency_s":6.974608}1317gitlab # [ 250.039402] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bae59a20b49d2270aa516e5","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bae59a20b49d2270aa516e5: done: 0.015107 sec","job_status":"done","queue_duration_s":6.974591,"scheduling_latency_s":6.974608,"redis_calls":13,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015107,"completed_at":"2026-09-13T13:03:46.121Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 250.079042] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d897096f11a09feb85abc28","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d897096f11a09feb85abc28: start","job_status":"start","queue_duration_s":6.990846,"scheduling_latency_s":6.990863}1319gitlab # [ 250.091539] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d897096f11a09feb85abc28","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d897096f11a09feb85abc28: done: 0.015652 sec","job_status":"done","queue_duration_s":6.990846,"scheduling_latency_s":6.990863,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010293,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015652,"completed_at":"2026-09-13T13:03:46.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 250.130689] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dbae91203ac938343830763","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dbae91203ac938343830763: start","job_status":"start","queue_duration_s":7.008,"scheduling_latency_s":7.008018}1321gitlab # [ 250.142964] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dbae91203ac938343830763","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dbae91203ac938343830763: done: 0.014835 sec","job_status":"done","queue_duration_s":7.008,"scheduling_latency_s":7.008018,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014835,"completed_at":"2026-09-13T13:03:46.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 250.183385] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22f0bb2c9c397fd7884024ba","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22f0bb2c9c397fd7884024ba: start","job_status":"start","queue_duration_s":7.023926,"scheduling_latency_s":7.023943}1323gitlab # [ 250.195985] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22f0bb2c9c397fd7884024ba","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22f0bb2c9c397fd7884024ba: done: 0.015035 sec","job_status":"done","queue_duration_s":7.023926,"scheduling_latency_s":7.023943,"redis_calls":13,"redis_duration_s":0.0015929999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015035,"completed_at":"2026-09-13T13:03:46.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 250.235062] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de8c8e9b07682acc7c1f5ae","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de8c8e9b07682acc7c1f5ae: start","job_status":"start","queue_duration_s":7.040031,"scheduling_latency_s":7.040048}1325gitlab # [ 250.247348] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de8c8e9b07682acc7c1f5ae","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de8c8e9b07682acc7c1f5ae: done: 0.01507 sec","job_status":"done","queue_duration_s":7.040031,"scheduling_latency_s":7.040048,"redis_calls":13,"redis_duration_s":0.001848,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01507,"completed_at":"2026-09-13T13:03:46.187Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 250.285618] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33d2d1cb8c974bec01819b6f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33d2d1cb8c974bec01819b6f: start","job_status":"start","queue_duration_s":7.056309,"scheduling_latency_s":7.056327}1327gitlab # [ 250.297702] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33d2d1cb8c974bec01819b6f","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33d2d1cb8c974bec01819b6f: done: 0.015351 sec","job_status":"done","queue_duration_s":7.056309,"scheduling_latency_s":7.056327,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015351,"completed_at":"2026-09-13T13:03:46.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 250.335431] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f78066aa309ad09f622bdee","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f78066aa309ad09f622bdee: start","job_status":"start","queue_duration_s":7.073188,"scheduling_latency_s":7.073205}1329gitlab # [ 250.347729] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f78066aa309ad09f622bdee","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f78066aa309ad09f622bdee: done: 0.015066 sec","job_status":"done","queue_duration_s":7.073188,"scheduling_latency_s":7.073205,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015066,"completed_at":"2026-09-13T13:03:46.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 250.387770] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e8cfe2e21e8edf8418a00b0","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e8cfe2e21e8edf8418a00b0: start","job_status":"start","queue_duration_s":7.089396,"scheduling_latency_s":7.089413}1331gitlab # [ 250.400118] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e8cfe2e21e8edf8418a00b0","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e8cfe2e21e8edf8418a00b0: done: 0.01514 sec","job_status":"done","queue_duration_s":7.089396,"scheduling_latency_s":7.089413,"redis_calls":13,"redis_duration_s":0.001909,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001377,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01514,"completed_at":"2026-09-13T13:03:46.236Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 250.438864] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"735e40a350e1c39e25ca9e64","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-735e40a350e1c39e25ca9e64: start","job_status":"start","queue_duration_s":7.105673,"scheduling_latency_s":7.10569}1333gitlab # [ 250.451302] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"735e40a350e1c39e25ca9e64","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-735e40a350e1c39e25ca9e64: done: 0.015371 sec","job_status":"done","queue_duration_s":7.105673,"scheduling_latency_s":7.10569,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015371,"completed_at":"2026-09-13T13:03:46.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 250.490481] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2180681f14b8cfe9f2ff760b","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2180681f14b8cfe9f2ff760b: start","job_status":"start","queue_duration_s":7.122201,"scheduling_latency_s":7.122219}1335gitlab # [ 250.503012] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2180681f14b8cfe9f2ff760b","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2180681f14b8cfe9f2ff760b: done: 0.015239 sec","job_status":"done","queue_duration_s":7.122201,"scheduling_latency_s":7.122219,"redis_calls":13,"redis_duration_s":0.001536,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000992,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015239,"completed_at":"2026-09-13T13:03:46.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 250.541493] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"713c8b22ed83bae4a141a799","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-713c8b22ed83bae4a141a799: start","job_status":"start","queue_duration_s":7.139193,"scheduling_latency_s":7.139211}1337gitlab # [ 250.553611] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"713c8b22ed83bae4a141a799","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-713c8b22ed83bae4a141a799: done: 0.015386 sec","job_status":"done","queue_duration_s":7.139193,"scheduling_latency_s":7.139211,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015386,"completed_at":"2026-09-13T13:03:46.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 250.592516] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d23be614c621993ae4da7b9","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d23be614c621993ae4da7b9: start","job_status":"start","queue_duration_s":7.155672,"scheduling_latency_s":7.15569}1339gitlab # [ 250.604631] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d23be614c621993ae4da7b9","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d23be614c621993ae4da7b9: done: 0.015818 sec","job_status":"done","queue_duration_s":7.155672,"scheduling_latency_s":7.15569,"redis_calls":13,"redis_duration_s":0.001728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001142,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015818,"completed_at":"2026-09-13T13:03:46.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 250.642769] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"714f04cafaeec8ca68ade5e2","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-714f04cafaeec8ca68ade5e2: start","job_status":"start","queue_duration_s":7.172619,"scheduling_latency_s":7.172636}1341gitlab # [ 250.654846] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"714f04cafaeec8ca68ade5e2","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-714f04cafaeec8ca68ade5e2: done: 0.015621 sec","job_status":"done","queue_duration_s":7.172619,"scheduling_latency_s":7.172636,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015621,"completed_at":"2026-09-13T13:03:46.320Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 250.693450] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.321Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"806093cd9dff02fec9b75ebd","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-806093cd9dff02fec9b75ebd: start","job_status":"start","queue_duration_s":7.189409,"scheduling_latency_s":7.189426}1343gitlab # [ 250.705638] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"806093cd9dff02fec9b75ebd","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-806093cd9dff02fec9b75ebd: done: 0.015565 sec","job_status":"done","queue_duration_s":7.189409,"scheduling_latency_s":7.189426,"redis_calls":13,"redis_duration_s":0.001443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015565,"completed_at":"2026-09-13T13:03:46.337Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 250.743982] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f44bc662c7ca5bb1f3ddd57","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f44bc662c7ca5bb1f3ddd57: start","job_status":"start","queue_duration_s":7.206088,"scheduling_latency_s":7.206106}1345gitlab # [ 250.756745] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f44bc662c7ca5bb1f3ddd57","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f44bc662c7ca5bb1f3ddd57: done: 0.015447 sec","job_status":"done","queue_duration_s":7.206088,"scheduling_latency_s":7.206106,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015447,"completed_at":"2026-09-13T13:03:46.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 250.796761] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a557ad32788bf917e25511c","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a557ad32788bf917e25511c: start","job_status":"start","queue_duration_s":7.222631,"scheduling_latency_s":7.222648}1347gitlab # [ 250.808986] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a557ad32788bf917e25511c","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a557ad32788bf917e25511c: done: 0.023165 sec","job_status":"done","queue_duration_s":7.222631,"scheduling_latency_s":7.222648,"redis_calls":13,"redis_duration_s":0.001485,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023165,"completed_at":"2026-09-13T13:03:46.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 250.848725] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d236b2b87684f02b277a887","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d236b2b87684f02b277a887: start","job_status":"start","queue_duration_s":7.246915,"scheduling_latency_s":7.246932}1349gitlab # [ 250.885594] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.395Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d236b2b87684f02b277a887","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d236b2b87684f02b277a887: done: 0.016404 sec","job_status":"done","queue_duration_s":7.246915,"scheduling_latency_s":7.246932,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016404,"completed_at":"2026-09-13T13:03:46.395Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 250.955668] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.396Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3410be86866bbadc628ded4d","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3410be86866bbadc628ded4d: start","job_status":"start","queue_duration_s":7.264461,"scheduling_latency_s":7.264478}1351gitlab # [ 250.969779] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3410be86866bbadc628ded4d","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3410be86866bbadc628ded4d: done: 0.015164 sec","job_status":"done","queue_duration_s":7.264461,"scheduling_latency_s":7.264478,"redis_calls":13,"redis_duration_s":0.0015220000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015164,"completed_at":"2026-09-13T13:03:46.411Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 251.010694] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.413Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d4c9562928ef8524bc118d6","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d4c9562928ef8524bc118d6: start","job_status":"start","queue_duration_s":7.280734,"scheduling_latency_s":7.280751}1353gitlab # [ 251.022820] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.428Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d4c9562928ef8524bc118d6","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d4c9562928ef8524bc118d6: done: 0.015112 sec","job_status":"done","queue_duration_s":7.280734,"scheduling_latency_s":7.280751,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015112,"completed_at":"2026-09-13T13:03:46.428Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 251.060870] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c92b9e5fd60665e25e36c401","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c92b9e5fd60665e25e36c401: start","job_status":"start","queue_duration_s":7.297011,"scheduling_latency_s":7.297028}1355gitlab # [ 251.073053] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.444Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c92b9e5fd60665e25e36c401","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c92b9e5fd60665e25e36c401: done: 0.014999 sec","job_status":"done","queue_duration_s":7.297011,"scheduling_latency_s":7.297028,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014999,"completed_at":"2026-09-13T13:03:46.444Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 251.111185] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8d4f53393e368fcc0a2e187","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8d4f53393e368fcc0a2e187: start","job_status":"start","queue_duration_s":7.313496,"scheduling_latency_s":7.313513}1357gitlab # [ 251.123458] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f8d4f53393e368fcc0a2e187","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f8d4f53393e368fcc0a2e187: done: 0.014882 sec","job_status":"done","queue_duration_s":7.313496,"scheduling_latency_s":7.313513,"redis_calls":13,"redis_duration_s":0.001469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014882,"completed_at":"2026-09-13T13:03:46.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 251.162109] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.462Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef6adc7ad35dfef2c61816f1","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef6adc7ad35dfef2c61816f1: start","job_status":"start","queue_duration_s":7.329628,"scheduling_latency_s":7.329645}1359gitlab # [ 251.174271] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef6adc7ad35dfef2c61816f1","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef6adc7ad35dfef2c61816f1: done: 0.015124 sec","job_status":"done","queue_duration_s":7.329628,"scheduling_latency_s":7.329645,"redis_calls":13,"redis_duration_s":0.001778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001224,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015124,"completed_at":"2026-09-13T13:03:46.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 251.213867] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.478Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c3de0201ee9e47bd3a8f105","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c3de0201ee9e47bd3a8f105: start","job_status":"start","queue_duration_s":7.345864,"scheduling_latency_s":7.34588}1361gitlab # [ 251.226439] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.492Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c3de0201ee9e47bd3a8f105","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c3de0201ee9e47bd3a8f105: done: 0.014733 sec","job_status":"done","queue_duration_s":7.345864,"scheduling_latency_s":7.34588,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010258,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014733,"completed_at":"2026-09-13T13:03:46.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 251.264469] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.494Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"840923e7bdc977c84f2e46e2","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-840923e7bdc977c84f2e46e2: start","job_status":"start","queue_duration_s":7.361958,"scheduling_latency_s":7.361976}1363gitlab # [ 251.276577] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.509Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"840923e7bdc977c84f2e46e2","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-840923e7bdc977c84f2e46e2: done: 0.015036 sec","job_status":"done","queue_duration_s":7.361958,"scheduling_latency_s":7.361976,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000959,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015036,"completed_at":"2026-09-13T13:03:46.509Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 251.314884] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.510Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66278631bffe99261f8695e7","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66278631bffe99261f8695e7: start","job_status":"start","queue_duration_s":7.378426,"scheduling_latency_s":7.378443}1365gitlab # [ 251.327167] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66278631bffe99261f8695e7","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66278631bffe99261f8695e7: done: 0.014763 sec","job_status":"done","queue_duration_s":7.378426,"scheduling_latency_s":7.378443,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014763,"completed_at":"2026-09-13T13:03:46.525Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 251.365392] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.526Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6fd446282a19a250352d28a","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6fd446282a19a250352d28a: start","job_status":"start","queue_duration_s":7.3943,"scheduling_latency_s":7.394317}1367gitlab # [ 251.377513] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6fd446282a19a250352d28a","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6fd446282a19a250352d28a: done: 0.014879 sec","job_status":"done","queue_duration_s":7.3943,"scheduling_latency_s":7.394317,"redis_calls":13,"redis_duration_s":0.001461,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014879,"completed_at":"2026-09-13T13:03:46.541Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 251.417039] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc3c2921f480d7ae53de194","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc3c2921f480d7ae53de194: start","job_status":"start","queue_duration_s":7.410302,"scheduling_latency_s":7.410318}1369gitlab # [ 251.429268] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.557Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc3c2921f480d7ae53de194","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc3c2921f480d7ae53de194: done: 0.014872 sec","job_status":"done","queue_duration_s":7.410302,"scheduling_latency_s":7.410318,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014872,"completed_at":"2026-09-13T13:03:46.557Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 251.468219] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d431e43bf39e006b6c6f9ee1","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d431e43bf39e006b6c6f9ee1: start","job_status":"start","queue_duration_s":7.426581,"scheduling_latency_s":7.426598}1371gitlab # [ 251.480488] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d431e43bf39e006b6c6f9ee1","created_at":"2026-09-13T13:03:38.322Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-7ec5a61ef8ff4012","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2597212b60604fb56f23de8c75c9640a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:39.132Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d431e43bf39e006b6c6f9ee1: done: 0.014675 sec","job_status":"done","queue_duration_s":7.426581,"scheduling_latency_s":7.426598,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014675,"completed_at":"2026-09-13T13:03:46.573Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 251.519252] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.574Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a4e6efac50c3eaa6f28fbe4b","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a4e6efac50c3eaa6f28fbe4b: start","job_status":"start","queue_duration_s":3.64842,"scheduling_latency_s":3.648437}1373gitlab # [ 251.531972] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.592Z","message":"No existing merge request to be cleaned up."}1374gitlab # [ 251.533755] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.607Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a4e6efac50c3eaa6f28fbe4b","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a4e6efac50c3eaa6f28fbe4b: done: 0.033125 sec","job_status":"done","queue_duration_s":3.64842,"scheduling_latency_s":3.648437,"redis_calls":13,"redis_duration_s":0.001872,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000316,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000658,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000898,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027404,"duration_s":0.033125,"completed_at":"2026-09-13T13:03:46.607Z","load_balancing_strategy":"primary","db_duration_s":0.006041,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1375gitlab # [ 251.573653] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.609Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"87048c638b0d99ca47aecc41","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-87048c638b0d99ca47aecc41: start","job_status":"start","queue_duration_s":3.682709,"scheduling_latency_s":3.682726}1376gitlab # [ 251.585951] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.619Z","message":"No existing merge request to be cleaned up."}1377gitlab # [ 251.587577] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.628Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"87048c638b0d99ca47aecc41","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-87048c638b0d99ca47aecc41: done: 0.019122 sec","job_status":"done","queue_duration_s":3.682709,"scheduling_latency_s":3.682726,"redis_calls":10,"redis_duration_s":0.001357,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000792,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019122,"completed_at":"2026-09-13T13:03:46.628Z","load_balancing_strategy":"primary","db_duration_s":0.002494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 251.627005] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.629Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f930b431cb7c20513ca5b818","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f930b431cb7c20513ca5b818: start","job_status":"start","queue_duration_s":3.702996,"scheduling_latency_s":3.703014}1379gitlab # [ 251.639395] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.640Z","message":"No existing merge request to be cleaned up."}1380gitlab # [ 251.641112] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.649Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f930b431cb7c20513ca5b818","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f930b431cb7c20513ca5b818: done: 0.01976 sec","job_status":"done","queue_duration_s":3.702996,"scheduling_latency_s":3.703014,"redis_calls":10,"redis_duration_s":0.001433,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01976,"completed_at":"2026-09-13T13:03:46.649Z","load_balancing_strategy":"primary","db_duration_s":0.002504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1381gitlab # [ 251.679527] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.650Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a79205619571fd587bbe9a17","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a79205619571fd587bbe9a17: start","job_status":"start","queue_duration_s":3.724264,"scheduling_latency_s":3.724282}1382gitlab # [ 251.691685] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.661Z","message":"No existing merge request to be cleaned up."}1383gitlab # [ 251.693832] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:03:46.670Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a79205619571fd587bbe9a17","created_at":"2026-09-13T13:03:42.838Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-75446b3b21d64c60","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"90e8d1c58a617745409bc3b732a20935","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:03:42.926Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a79205619571fd587bbe9a17: done: 0.019624 sec","job_status":"done","queue_duration_s":3.724264,"scheduling_latency_s":3.724282,"redis_calls":10,"redis_duration_s":0.001366,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00058,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000786,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019624,"completed_at":"2026-09-13T13:03:46.670Z","load_balancing_strategy":"primary","db_duration_s":0.002516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 267.291064] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.177Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a029b5bbd534fa841bd5b6ad","created_at":"2026-09-13T13:04:04.159Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d6c0cfb475b5fbb8ebe5c02cff2f789","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.172Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a029b5bbd534fa841bd5b6ad: start","job_status":"start","queue_duration_s":0.004194,"scheduling_latency_s":0.004249}1385gitlab # [ 267.475305] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.363Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"a029b5bbd534fa841bd5b6ad","created_at":"2026-09-13T13:04:04.159Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d6c0cfb475b5fbb8ebe5c02cff2f789","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.172Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-a029b5bbd534fa841bd5b6ad: done: 0.186151 sec","job_status":"done","queue_duration_s":0.004194,"scheduling_latency_s":0.004249,"redis_calls":9,"redis_duration_s":0.054804,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012482,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.042322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00235346,"exclusive_lock_hold_duration_s":0.04059302,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.186151,"completed_at":"2026-09-13T13:04:04.362Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033376,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 267.575790] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"074490620972b9ca8a74d34e","created_at":"2026-09-13T13:04:04.199Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06d87f1b7575cea01bbf90d1ba3f3361","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.219Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-074490620972b9ca8a74d34e: start","job_status":"start","queue_duration_s":0.184691,"scheduling_latency_s":0.184739}1387gitlab # [ 267.709569] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.597Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"074490620972b9ca8a74d34e","created_at":"2026-09-13T13:04:04.199Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06d87f1b7575cea01bbf90d1ba3f3361","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.219Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-074490620972b9ca8a74d34e: done: 0.193387 sec","job_status":"done","queue_duration_s":0.184691,"scheduling_latency_s":0.184739,"redis_calls":9,"redis_duration_s":0.061169999999999995,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.039066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.022104,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00388618,"exclusive_lock_hold_duration_s":0.02235268,"duration_s":0.193387,"completed_at":"2026-09-13T13:04:04.597Z","load_balancing_strategy":"primary","db_duration_s":0.016456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1388gitlab # [ 267.800193] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.601Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c5a504304085b418744613b3","created_at":"2026-09-13T13:04:04.236Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bac855752e939bd7dedfa7138aa043f6","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.292Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c5a504304085b418744613b3: start","job_status":"start","queue_duration_s":0.309763,"scheduling_latency_s":0.309817}1389gitlab # [ 267.839577] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c5a504304085b418744613b3","created_at":"2026-09-13T13:04:04.236Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bac855752e939bd7dedfa7138aa043f6","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.292Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c5a504304085b418744613b3: done: 0.125335 sec","job_status":"done","queue_duration_s":0.309763,"scheduling_latency_s":0.309817,"redis_calls":10,"redis_duration_s":0.005244,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000678,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00071108,"duration_s":0.125335,"completed_at":"2026-09-13T13:04:04.727Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031694,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1390gitlab # [ 267.898824] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.732Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"432473e07a8946214b323a3c","created_at":"2026-09-13T13:04:04.342Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fe3b37486d6305ecde380b6c2391956","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.357Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-432473e07a8946214b323a3c: start","job_status":"start","queue_duration_s":0.375072,"scheduling_latency_s":0.375128}1391gitlab # [ 267.977506] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"432473e07a8946214b323a3c","created_at":"2026-09-13T13:04:04.342Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2fe3b37486d6305ecde380b6c2391956","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.357Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-432473e07a8946214b323a3c: done: 0.133687 sec","job_status":"done","queue_duration_s":0.375072,"scheduling_latency_s":0.375128,"redis_calls":10,"redis_duration_s":0.009323,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004637,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004686,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.06463666,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.133687,"completed_at":"2026-09-13T13:04:04.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.059076,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1392gitlab # [ 268.039847] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:04.898Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"75ae0d0a4035515b506ff5ef","created_at":"2026-09-13T13:04:04.399Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.407Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-75ae0d0a4035515b506ff5ef: start","job_status":"start","queue_duration_s":0.490702,"scheduling_latency_s":0.490753}1393gitlab # [ 268.217651] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.106Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"75ae0d0a4035515b506ff5ef","created_at":"2026-09-13T13:04:04.399Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.407Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-75ae0d0a4035515b506ff5ef: done: 0.207623 sec","job_status":"done","queue_duration_s":0.490702,"scheduling_latency_s":0.490753,"redis_calls":60,"redis_duration_s":0.018786999999999998,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.000682,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.017158,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.119926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.207623,"completed_at":"2026-09-13T13:04:05.106Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1394gitlab # [ 268.309156] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.107Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"bb80a3d1cc80c052a31ec413","created_at":"2026-09-13T13:04:04.421Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"970a7d3489f93a17826a8e4c26edad3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.427Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-bb80a3d1cc80c052a31ec413: start","job_status":"start","queue_duration_s":0.679821,"scheduling_latency_s":0.679839}1395gitlab # [ 268.327317] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.122Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"bb80a3d1cc80c052a31ec413","created_at":"2026-09-13T13:04:04.421Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"970a7d3489f93a17826a8e4c26edad3e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.427Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-bb80a3d1cc80c052a31ec413: done: 0.014324 sec","job_status":"done","queue_duration_s":0.679821,"scheduling_latency_s":0.679839,"redis_calls":7,"redis_duration_s":0.001654,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000458,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014324,"completed_at":"2026-09-13T13:04:05.122Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 268.369397] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f94b59b8dc3de97aabcdb340","created_at":"2026-09-13T13:04:04.445Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03f3162fd5cfb7139392207822f00b69","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.450Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f94b59b8dc3de97aabcdb340: start","job_status":"start","queue_duration_s":0.673239,"scheduling_latency_s":0.673255}1397gitlab # [ 268.572798] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"f94b59b8dc3de97aabcdb340","created_at":"2026-09-13T13:04:04.445Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"03f3162fd5cfb7139392207822f00b69","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.450Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-f94b59b8dc3de97aabcdb340: done: 0.337738 sec","job_status":"done","queue_duration_s":0.673239,"scheduling_latency_s":0.673255,"redis_calls":562,"redis_duration_s":0.062816,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.06173,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.311927,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.337738,"completed_at":"2026-09-13T13:04:05.461Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 268.661590] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.462Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"3431717ff3c91e13dcc9b55c","created_at":"2026-09-13T13:04:04.459Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e59e15ea17816889802b1e318b921812","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.481Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-3431717ff3c91e13dcc9b55c: start","job_status":"start","queue_duration_s":0.981321,"scheduling_latency_s":0.981343}1399gitlab # [ 268.680788] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.483Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"3431717ff3c91e13dcc9b55c","created_at":"2026-09-13T13:04:04.459Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e59e15ea17816889802b1e318b921812","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.481Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-3431717ff3c91e13dcc9b55c: done: 0.020494 sec","job_status":"done","queue_duration_s":0.981321,"scheduling_latency_s":0.981343,"redis_calls":12,"redis_duration_s":0.0026,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000543,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000492,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031416,"duration_s":0.020494,"completed_at":"2026-09-13T13:04:05.483Z","load_balancing_strategy":"primary","db_duration_s":0.000694,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 268.732643] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9287b26a3501ea4032bd985c","created_at":"2026-09-13T13:04:04.490Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9680ee25ade22fe79a6578da9e83f537","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.495Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9287b26a3501ea4032bd985c: start","job_status":"start","queue_duration_s":0.988891,"scheduling_latency_s":0.988908}1401gitlab # [ 268.747090] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.492Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1402gitlab # [ 268.750524] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.497Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9287b26a3501ea4032bd985c","created_at":"2026-09-13T13:04:04.490Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9680ee25ade22fe79a6578da9e83f537","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.495Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9287b26a3501ea4032bd985c: done: 0.013096 sec","job_status":"done","queue_duration_s":0.988891,"scheduling_latency_s":0.988908,"redis_calls":7,"redis_duration_s":0.0016070000000000001,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00044,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013096,"completed_at":"2026-09-13T13:04:05.497Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1403gitlab # [ 268.791965] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.499Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dce9be47d76b10a70250a2f0","created_at":"2026-09-13T13:04:04.502Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e264f114cf3813b30a8d528c059823a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T13:04:04.509Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dce9be47d76b10a70250a2f0: start","job_status":"start","queue_duration_s":0.990182,"scheduling_latency_s":0.990198}1404gitlab # [ 268.805254] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.507Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1405gitlab # [ 268.808495] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.512Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"dce9be47d76b10a70250a2f0","created_at":"2026-09-13T13:04:04.502Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e264f114cf3813b30a8d528c059823a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T13:04:04.509Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-dce9be47d76b10a70250a2f0: done: 0.013306 sec","job_status":"done","queue_duration_s":0.990182,"scheduling_latency_s":0.990198,"redis_calls":7,"redis_duration_s":0.001731,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001235,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013306,"completed_at":"2026-09-13T13:04:05.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 268.847428] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"724a6f65aa8d23fe732e9325","created_at":"2026-09-13T13:04:04.521Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.526Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-724a6f65aa8d23fe732e9325: start","job_status":"start","queue_duration_s":0.98763,"scheduling_latency_s":0.987646}1407gitlab # [ 268.860900] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.541Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"724a6f65aa8d23fe732e9325","created_at":"2026-09-13T13:04:04.521Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.526Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-724a6f65aa8d23fe732e9325: done: 0.027884 sec","job_status":"done","queue_duration_s":0.98763,"scheduling_latency_s":0.987646,"redis_calls":14,"redis_duration_s":0.003307,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000375,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002148,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027884,"completed_at":"2026-09-13T13:04:05.541Z","load_balancing_strategy":"primary","db_duration_s":0.00292,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 268.901444] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.543Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fa58a19fec6f4fee93976882","created_at":"2026-09-13T13:04:04.529Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7289996d3fde2119f671faaa01c49034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T13:04:04.544Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fa58a19fec6f4fee93976882: start","job_status":"start","queue_duration_s":0.998735,"scheduling_latency_s":0.998752}1409gitlab # [ 268.914626] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"fa58a19fec6f4fee93976882","created_at":"2026-09-13T13:04:04.529Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7289996d3fde2119f671faaa01c49034","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T13:04:04.544Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-fa58a19fec6f4fee93976882: done: 0.027459 sec","job_status":"done","queue_duration_s":0.998735,"scheduling_latency_s":0.998752,"redis_calls":7,"redis_duration_s":0.002375,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001692,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027459,"completed_at":"2026-09-13T13:04:05.570Z","load_balancing_strategy":"primary","db_duration_s":0.004457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1410gitlab # [ 268.953499] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.572Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"21658714f2c6f72d243cc699","created_at":"2026-09-13T13:04:04.561Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92d4b9bc85143e38dff03689cd519af2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.571Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-21658714f2c6f72d243cc699: start","job_status":"start","queue_duration_s":1.000613,"scheduling_latency_s":1.000631}1411gitlab # [ 268.967730] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.615Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"21658714f2c6f72d243cc699","created_at":"2026-09-13T13:04:04.561Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"92d4b9bc85143e38dff03689cd519af2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.571Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-21658714f2c6f72d243cc699: done: 0.04267 sec","job_status":"done","queue_duration_s":1.000613,"scheduling_latency_s":1.000631,"redis_calls":7,"redis_duration_s":0.002022,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015589,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04267,"completed_at":"2026-09-13T13:04:05.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1412gitlab # [ 269.008843] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.618Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a1829c86f0b817996818ed5c","created_at":"2026-09-13T13:04:04.588Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d5353f81826506b834ecd455eed24b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T13:04:04.594Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a1829c86f0b817996818ed5c: start","job_status":"start","queue_duration_s":1.024336,"scheduling_latency_s":1.024381}1413gitlab # [ 269.022001] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.637Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1414gitlab # [ 269.025661] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.649Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a1829c86f0b817996818ed5c","created_at":"2026-09-13T13:04:04.588Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d5353f81826506b834ecd455eed24b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T13:04:04.594Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a1829c86f0b817996818ed5c: done: 0.030754 sec","job_status":"done","queue_duration_s":1.024336,"scheduling_latency_s":1.024381,"redis_calls":7,"redis_duration_s":0.002623,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002049,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000574,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030754,"completed_at":"2026-09-13T13:04:05.649Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1415gitlab # [ 269.064813] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"fe1b79f195e54cfdae8182d7","created_at":"2026-09-13T13:04:04.636Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b3d010e0cef610ac95e754a40e44356","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.672Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-fe1b79f195e54cfdae8182d7: start","job_status":"start","queue_duration_s":0.979223,"scheduling_latency_s":0.979259}1416gitlab # [ 269.078872] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.698Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"fe1b79f195e54cfdae8182d7","created_at":"2026-09-13T13:04:04.636Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b3d010e0cef610ac95e754a40e44356","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.672Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-fe1b79f195e54cfdae8182d7: done: 0.047272 sec","job_status":"done","queue_duration_s":0.979223,"scheduling_latency_s":0.979259,"redis_calls":12,"redis_duration_s":0.003996,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000666,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001484,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00093502,"exclusive_lock_hold_duration_s":0.01255322,"duration_s":0.047272,"completed_at":"2026-09-13T13:04:05.698Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007844,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1417gitlab # [ 269.120609] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.701Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"bd55e568fec5b2c0b0d5da46","created_at":"2026-09-13T13:04:04.714Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdf393dd3a2a01a693d0be6ff67c0511","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.721Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-bd55e568fec5b2c0b0d5da46: start","job_status":"start","queue_duration_s":0.979405,"scheduling_latency_s":0.979433}1418gitlab # [ 269.134973] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.722Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"bd55e568fec5b2c0b0d5da46","created_at":"2026-09-13T13:04:04.714Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bdf393dd3a2a01a693d0be6ff67c0511","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.721Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-bd55e568fec5b2c0b0d5da46: done: 0.021043 sec","job_status":"done","queue_duration_s":0.979405,"scheduling_latency_s":0.979433,"redis_calls":7,"redis_duration_s":0.002016,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021043,"completed_at":"2026-09-13T13:04:05.722Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 269.175148] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.724Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"60cd061ef7098f178a0ea97b","created_at":"2026-09-13T13:04:04.751Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"7d450c1576168e99ee7bf250497bd484","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.772Z","job_size_bytes":3,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-60cd061ef7098f178a0ea97b: start","job_status":"start","queue_duration_s":0.951691,"scheduling_latency_s":0.951715}1420gitlab # [ 269.190513] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.747Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"60cd061ef7098f178a0ea97b","created_at":"2026-09-13T13:04:04.751Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"7d450c1576168e99ee7bf250497bd484","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.772Z","job_size_bytes":3,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-60cd061ef7098f178a0ea97b: done: 0.023324 sec","job_status":"done","queue_duration_s":0.951691,"scheduling_latency_s":0.951715,"redis_calls":10,"redis_duration_s":0.002722,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000606,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001586,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00053,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003809,"duration_s":0.023324,"completed_at":"2026-09-13T13:04:05.747Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000719,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 269.232424] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.749Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"572d7490b526df05e36dc4fb","created_at":"2026-09-13T13:04:04.796Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37ef8cb9b3ee61627e105a15bcea3e23","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.799Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-572d7490b526df05e36dc4fb: start","job_status":"start","queue_duration_s":0.949463,"scheduling_latency_s":0.949506}1422gitlab # [ 269.246839] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.758Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1423gitlab # [ 269.250676] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.764Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"572d7490b526df05e36dc4fb","created_at":"2026-09-13T13:04:04.796Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37ef8cb9b3ee61627e105a15bcea3e23","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.799Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-572d7490b526df05e36dc4fb: done: 0.015314 sec","job_status":"done","queue_duration_s":0.949463,"scheduling_latency_s":0.949506,"redis_calls":7,"redis_duration_s":0.0017499999999999998,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001259,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015314,"completed_at":"2026-09-13T13:04:05.764Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 269.291582] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.766Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4d9b3f2ab3a0238c4335b90b","created_at":"2026-09-13T13:04:04.809Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8078e4a1212ed32c092a472e4d171655","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.812Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4d9b3f2ab3a0238c4335b90b: start","job_status":"start","queue_duration_s":0.953458,"scheduling_latency_s":0.953478}1425gitlab # [ 269.305131] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.805Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"4d9b3f2ab3a0238c4335b90b","created_at":"2026-09-13T13:04:04.809Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8078e4a1212ed32c092a472e4d171655","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.812Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-4d9b3f2ab3a0238c4335b90b: done: 0.039564 sec","job_status":"done","queue_duration_s":0.953458,"scheduling_latency_s":0.953478,"redis_calls":7,"redis_duration_s":0.002293,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.039564,"completed_at":"2026-09-13T13:04:05.805Z","load_balancing_strategy":"primary","db_duration_s":0.020946,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 269.345318] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.807Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9b71765b10e69a0f74f4f351","created_at":"2026-09-13T13:04:04.817Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7c5c4459e9448b256427eae259d3394","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.819Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9b71765b10e69a0f74f4f351: start","job_status":"start","queue_duration_s":0.988224,"scheduling_latency_s":0.988248}1427gitlab # [ 269.359439] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.860Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"9b71765b10e69a0f74f4f351","created_at":"2026-09-13T13:04:04.817Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7c5c4459e9448b256427eae259d3394","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.819Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-9b71765b10e69a0f74f4f351: done: 0.052466 sec","job_status":"done","queue_duration_s":0.988224,"scheduling_latency_s":0.988248,"redis_calls":9,"redis_duration_s":0.002207,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001141,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060118,"exclusive_lock_hold_duration_s":0.03335928,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.052466,"completed_at":"2026-09-13T13:04:05.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1428gitlab # [ 269.402368] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.861Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9662e36fb2f60372e30f274c","created_at":"2026-09-13T13:04:04.833Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e52e26acd267838f40d7168074c4b5f2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.840Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9662e36fb2f60372e30f274c: start","job_status":"start","queue_duration_s":1.02099,"scheduling_latency_s":1.021013}1429gitlab # [ 269.416139] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9662e36fb2f60372e30f274c","created_at":"2026-09-13T13:04:04.833Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e52e26acd267838f40d7168074c4b5f2","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.840Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9662e36fb2f60372e30f274c: done: 0.01858 sec","job_status":"done","queue_duration_s":1.02099,"scheduling_latency_s":1.021013,"redis_calls":7,"redis_duration_s":0.0012230000000000001,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01858,"completed_at":"2026-09-13T13:04:05.880Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001666,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 269.455124] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.881Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"21c76d6ddc90e5e34dc461f3","created_at":"2026-09-13T13:04:04.854Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"615b05c70195dda458305e3ac260bb19","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.862Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-21c76d6ddc90e5e34dc461f3: start","job_status":"start","queue_duration_s":1.019776,"scheduling_latency_s":1.019799}1431gitlab # [ 269.469060] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.890Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1432gitlab # [ 269.472158] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"21c76d6ddc90e5e34dc461f3","created_at":"2026-09-13T13:04:04.854Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"615b05c70195dda458305e3ac260bb19","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.862Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-21c76d6ddc90e5e34dc461f3: done: 0.014551 sec","job_status":"done","queue_duration_s":1.019776,"scheduling_latency_s":1.019799,"redis_calls":7,"redis_duration_s":0.001157,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000825,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014551,"completed_at":"2026-09-13T13:04:05.896Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1433gitlab # [ 269.511309] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:05.897Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"183a0e77a4fc3074b0ee331c","created_at":"2026-09-13T13:04:04.880Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62cf3888adba81c47e6a2c4650be2d46","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.883Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-183a0e77a4fc3074b0ee331c: start","job_status":"start","queue_duration_s":1.013672,"scheduling_latency_s":1.013691}1434gitlab # [ 269.525188] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.154Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"183a0e77a4fc3074b0ee331c","created_at":"2026-09-13T13:04:04.880Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62cf3888adba81c47e6a2c4650be2d46","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.883Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-183a0e77a4fc3074b0ee331c: done: 0.257203 sec","job_status":"done","queue_duration_s":1.013672,"scheduling_latency_s":1.013691,"redis_calls":11,"redis_duration_s":0.001708,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000662,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.053,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.190809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050232,"exclusive_lock_hold_duration_s":0.23932138,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.257203,"completed_at":"2026-09-13T13:04:06.154Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.078161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 269.569245] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.156Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"628fbf03901c0c972c3ce701","created_at":"2026-09-13T13:04:04.888Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef355c46a9e4e9850fe97383c74f9ce1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.894Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-628fbf03901c0c972c3ce701: start","job_status":"start","queue_duration_s":1.261692,"scheduling_latency_s":1.261709}1436gitlab # [ 269.583316] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"628fbf03901c0c972c3ce701","created_at":"2026-09-13T13:04:04.888Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ef355c46a9e4e9850fe97383c74f9ce1","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.894Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-628fbf03901c0c972c3ce701: done: 0.016142 sec","job_status":"done","queue_duration_s":1.261692,"scheduling_latency_s":1.261709,"redis_calls":9,"redis_duration_s":0.0012439999999999999,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000572,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004284,"exclusive_lock_hold_duration_s":0.00332664,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.016142,"completed_at":"2026-09-13T13:04:06.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001717,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 269.625412] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.173Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0d84c99cf85af53006e9f287","created_at":"2026-09-13T13:04:04.908Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed2ad9674ddd1778e6f4a9b1c3bd5007","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.912Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0d84c99cf85af53006e9f287: start","job_status":"start","queue_duration_s":1.260333,"scheduling_latency_s":1.260349}1438gitlab # [ 269.639649] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.180Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1439gitlab # [ 269.643363] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.185Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0d84c99cf85af53006e9f287","created_at":"2026-09-13T13:04:04.908Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-fd6c1165a4264937","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed2ad9674ddd1778e6f4a9b1c3bd5007","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T13:04:04.912Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0d84c99cf85af53006e9f287: done: 0.012107 sec","job_status":"done","queue_duration_s":1.260333,"scheduling_latency_s":1.260349,"redis_calls":7,"redis_duration_s":0.0009029999999999999,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000655,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012107,"completed_at":"2026-09-13T13:04:06.185Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1440gitlab # [ 269.683522] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fd085fcbf608e6079f0be12","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fd085fcbf608e6079f0be12: start","job_status":"start","queue_duration_s":1.088112,"scheduling_latency_s":1.088128}1441gitlab # [ 269.696027] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fd085fcbf608e6079f0be12","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fd085fcbf608e6079f0be12: done: 0.014462 sec","job_status":"done","queue_duration_s":1.088112,"scheduling_latency_s":1.088128,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014462,"completed_at":"2026-09-13T13:04:06.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 269.734312] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c3f22f11301b0f6e83d9523","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c3f22f11301b0f6e83d9523: start","job_status":"start","queue_duration_s":1.10371,"scheduling_latency_s":1.103727}1443gitlab # [ 269.746543] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c3f22f11301b0f6e83d9523","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c3f22f11301b0f6e83d9523: done: 0.014311 sec","job_status":"done","queue_duration_s":1.10371,"scheduling_latency_s":1.103727,"redis_calls":13,"redis_duration_s":0.0014299999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014311,"completed_at":"2026-09-13T13:04:06.216Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1444gitlab # [ 269.784917] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22f8fdea474d319f8db05abe","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22f8fdea474d319f8db05abe: start","job_status":"start","queue_duration_s":1.119145,"scheduling_latency_s":1.119162}1445gitlab # [ 269.797941] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22f8fdea474d319f8db05abe","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22f8fdea474d319f8db05abe: done: 0.013976 sec","job_status":"done","queue_duration_s":1.119145,"scheduling_latency_s":1.119162,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013976,"completed_at":"2026-09-13T13:04:06.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1446gitlab # [ 269.836288] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"204e762fef0d55f81db3fb81","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-204e762fef0d55f81db3fb81: start","job_status":"start","queue_duration_s":1.134189,"scheduling_latency_s":1.134205}1447gitlab # [ 269.848393] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"204e762fef0d55f81db3fb81","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-204e762fef0d55f81db3fb81: done: 0.014226 sec","job_status":"done","queue_duration_s":1.134189,"scheduling_latency_s":1.134205,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014226,"completed_at":"2026-09-13T13:04:06.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 269.886585] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8f7889203b3d7413667aad2","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8f7889203b3d7413667aad2: start","job_status":"start","queue_duration_s":1.149497,"scheduling_latency_s":1.149514}1449gitlab # [ 269.898851] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c8f7889203b3d7413667aad2","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c8f7889203b3d7413667aad2: done: 0.014082 sec","job_status":"done","queue_duration_s":1.149497,"scheduling_latency_s":1.149514,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009706,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014082,"completed_at":"2026-09-13T13:04:06.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1450gitlab # [ 269.937179] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34088c56950df8155b0756d1","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34088c56950df8155b0756d1: start","job_status":"start","queue_duration_s":1.164646,"scheduling_latency_s":1.164663}1451gitlab # [ 269.949371] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34088c56950df8155b0756d1","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34088c56950df8155b0756d1: done: 0.014073 sec","job_status":"done","queue_duration_s":1.164646,"scheduling_latency_s":1.164663,"redis_calls":13,"redis_duration_s":0.0014750000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014073,"completed_at":"2026-09-13T13:04:06.277Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1452gitlab # [ 269.988669] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.278Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c0ac364ea3404d498eb7cdb","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c0ac364ea3404d498eb7cdb: start","job_status":"start","queue_duration_s":1.179839,"scheduling_latency_s":1.179856}1453gitlab # [ 270.001943] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c0ac364ea3404d498eb7cdb","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c0ac364ea3404d498eb7cdb: done: 0.014089 sec","job_status":"done","queue_duration_s":1.179839,"scheduling_latency_s":1.179856,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009744,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014089,"completed_at":"2026-09-13T13:04:06.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1454gitlab # [ 270.040802] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fff281e71b9017960dbcfd49","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fff281e71b9017960dbcfd49: start","job_status":"start","queue_duration_s":1.19502,"scheduling_latency_s":1.195037}1455gitlab # [ 270.053001] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.307Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fff281e71b9017960dbcfd49","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fff281e71b9017960dbcfd49: done: 0.014178 sec","job_status":"done","queue_duration_s":1.19502,"scheduling_latency_s":1.195037,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014178,"completed_at":"2026-09-13T13:04:06.307Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1456gitlab # [ 270.092280] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd3f68a4adb4a80c7ba68260","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd3f68a4adb4a80c7ba68260: start","job_status":"start","queue_duration_s":1.210295,"scheduling_latency_s":1.210312}1457gitlab # [ 270.104558] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd3f68a4adb4a80c7ba68260","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd3f68a4adb4a80c7ba68260: done: 0.014366 sec","job_status":"done","queue_duration_s":1.210295,"scheduling_latency_s":1.210312,"redis_calls":13,"redis_duration_s":0.001665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014366,"completed_at":"2026-09-13T13:04:06.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1458gitlab # [ 270.143314] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d1c9fbfb23f6200c24facc3","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d1c9fbfb23f6200c24facc3: start","job_status":"start","queue_duration_s":1.225769,"scheduling_latency_s":1.225786}1459gitlab # [ 270.155766] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.338Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d1c9fbfb23f6200c24facc3","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d1c9fbfb23f6200c24facc3: done: 0.014333 sec","job_status":"done","queue_duration_s":1.225769,"scheduling_latency_s":1.225786,"redis_calls":13,"redis_duration_s":0.001411,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014333,"completed_at":"2026-09-13T13:04:06.338Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1460gitlab # [ 270.194858] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.340Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c938e68e0e204fdaf1ab38ad","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c938e68e0e204fdaf1ab38ad: start","job_status":"start","queue_duration_s":1.241548,"scheduling_latency_s":1.241565}1461gitlab # [ 270.208070] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c938e68e0e204fdaf1ab38ad","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c938e68e0e204fdaf1ab38ad: done: 0.014007 sec","job_status":"done","queue_duration_s":1.241548,"scheduling_latency_s":1.241565,"redis_calls":13,"redis_duration_s":0.001431,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014007,"completed_at":"2026-09-13T13:04:06.353Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1462gitlab # [ 270.246947] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa3e36b05ed168603ad6041","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa3e36b05ed168603ad6041: start","job_status":"start","queue_duration_s":1.256637,"scheduling_latency_s":1.256653}1463gitlab # [ 270.259505] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3aa3e36b05ed168603ad6041","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3aa3e36b05ed168603ad6041: done: 0.014262 sec","job_status":"done","queue_duration_s":1.256637,"scheduling_latency_s":1.256653,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014262,"completed_at":"2026-09-13T13:04:06.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1464gitlab # [ 270.298174] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f801f31d230b0f6d502abd8d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f801f31d230b0f6d502abd8d: start","job_status":"start","queue_duration_s":1.27196,"scheduling_latency_s":1.271978}1465gitlab # [ 270.310313] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f801f31d230b0f6d502abd8d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f801f31d230b0f6d502abd8d: done: 0.013961 sec","job_status":"done","queue_duration_s":1.27196,"scheduling_latency_s":1.271978,"redis_calls":13,"redis_duration_s":0.0013800000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013961,"completed_at":"2026-09-13T13:04:06.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1466gitlab # [ 270.349676] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa5474f2736e71ff92632059","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa5474f2736e71ff92632059: start","job_status":"start","queue_duration_s":1.287,"scheduling_latency_s":1.287017}1467gitlab # [ 270.361893] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.399Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa5474f2736e71ff92632059","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa5474f2736e71ff92632059: done: 0.014005 sec","job_status":"done","queue_duration_s":1.287,"scheduling_latency_s":1.287017,"redis_calls":13,"redis_duration_s":0.0014030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014005,"completed_at":"2026-09-13T13:04:06.399Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1468gitlab # [ 270.400572] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"854a4f1632950cb53ce4d837","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-854a4f1632950cb53ce4d837: start","job_status":"start","queue_duration_s":1.302091,"scheduling_latency_s":1.302143}1469gitlab # [ 270.413953] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.414Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"854a4f1632950cb53ce4d837","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-854a4f1632950cb53ce4d837: done: 0.014127 sec","job_status":"done","queue_duration_s":1.302091,"scheduling_latency_s":1.302143,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014127,"completed_at":"2026-09-13T13:04:06.414Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1470gitlab # [ 270.452534] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"def2755eff28b70ea9398071","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-def2755eff28b70ea9398071: start","job_status":"start","queue_duration_s":1.317321,"scheduling_latency_s":1.317338}1471gitlab # [ 270.464742] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.430Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"def2755eff28b70ea9398071","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-def2755eff28b70ea9398071: done: 0.014288 sec","job_status":"done","queue_duration_s":1.317321,"scheduling_latency_s":1.317338,"redis_calls":13,"redis_duration_s":0.001408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014288,"completed_at":"2026-09-13T13:04:06.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1472gitlab # [ 270.503684] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43804243124a2b72b3341d41","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43804243124a2b72b3341d41: start","job_status":"start","queue_duration_s":1.332631,"scheduling_latency_s":1.332647}1473gitlab # [ 270.515909] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.445Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43804243124a2b72b3341d41","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43804243124a2b72b3341d41: done: 0.014378 sec","job_status":"done","queue_duration_s":1.332631,"scheduling_latency_s":1.332647,"redis_calls":13,"redis_duration_s":0.0014320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014378,"completed_at":"2026-09-13T13:04:06.445Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1474gitlab # [ 270.555342] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.446Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c844157127c424bb4db2703","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c844157127c424bb4db2703: start","job_status":"start","queue_duration_s":1.348089,"scheduling_latency_s":1.348106}1475gitlab # [ 270.567568] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c844157127c424bb4db2703","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c844157127c424bb4db2703: done: 0.014023 sec","job_status":"done","queue_duration_s":1.348089,"scheduling_latency_s":1.348106,"redis_calls":13,"redis_duration_s":0.0013909999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014023,"completed_at":"2026-09-13T13:04:06.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1476gitlab # [ 270.606625] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1dc5a8343fd4a83a8ed61790","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1dc5a8343fd4a83a8ed61790: start","job_status":"start","queue_duration_s":1.363211,"scheduling_latency_s":1.363228}1477gitlab # [ 270.620143] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1dc5a8343fd4a83a8ed61790","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1dc5a8343fd4a83a8ed61790: done: 0.013951 sec","job_status":"done","queue_duration_s":1.363211,"scheduling_latency_s":1.363228,"redis_calls":13,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013951,"completed_at":"2026-09-13T13:04:06.475Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1478gitlab # [ 270.658832] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adfbb25487e3714e9c6b9b2a","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adfbb25487e3714e9c6b9b2a: start","job_status":"start","queue_duration_s":1.378238,"scheduling_latency_s":1.378255}1479gitlab # [ 270.671296] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"adfbb25487e3714e9c6b9b2a","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-adfbb25487e3714e9c6b9b2a: done: 0.014152 sec","job_status":"done","queue_duration_s":1.378238,"scheduling_latency_s":1.378255,"redis_calls":13,"redis_duration_s":0.001466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014152,"completed_at":"2026-09-13T13:04:06.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1480gitlab # [ 270.710118] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5a8ffb9bac8c07211603a53","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5a8ffb9bac8c07211603a53: start","job_status":"start","queue_duration_s":1.393447,"scheduling_latency_s":1.393463}1481gitlab # [ 270.722195] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5a8ffb9bac8c07211603a53","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5a8ffb9bac8c07211603a53: done: 0.013967 sec","job_status":"done","queue_duration_s":1.393447,"scheduling_latency_s":1.393463,"redis_calls":13,"redis_duration_s":0.001377,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013967,"completed_at":"2026-09-13T13:04:06.505Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1482gitlab # [ 270.760892] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e55f561b91cd318bd2894fa","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e55f561b91cd318bd2894fa: start","job_status":"start","queue_duration_s":1.408478,"scheduling_latency_s":1.408495}1483gitlab # [ 270.773092] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e55f561b91cd318bd2894fa","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e55f561b91cd318bd2894fa: done: 0.014385 sec","job_status":"done","queue_duration_s":1.408478,"scheduling_latency_s":1.408495,"redis_calls":13,"redis_duration_s":0.001408,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014385,"completed_at":"2026-09-13T13:04:06.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1484gitlab # [ 270.810892] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5615b6b67ac48bbddb737d9","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5615b6b67ac48bbddb737d9: start","job_status":"start","queue_duration_s":1.423944,"scheduling_latency_s":1.42396}1485gitlab # [ 270.824137] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d5615b6b67ac48bbddb737d9","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d5615b6b67ac48bbddb737d9: done: 0.014326 sec","job_status":"done","queue_duration_s":1.423944,"scheduling_latency_s":1.42396,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014326,"completed_at":"2026-09-13T13:04:06.536Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1486gitlab # [ 270.861828] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a18afa6da4eeb9a2382b9650","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a18afa6da4eeb9a2382b9650: start","job_status":"start","queue_duration_s":1.43937,"scheduling_latency_s":1.439386}1487gitlab # [ 270.873634] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a18afa6da4eeb9a2382b9650","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a18afa6da4eeb9a2382b9650: done: 0.013866 sec","job_status":"done","queue_duration_s":1.43937,"scheduling_latency_s":1.439386,"redis_calls":13,"redis_duration_s":0.001375,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013866,"completed_at":"2026-09-13T13:04:06.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1488gitlab # [ 270.911474] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c8cca093b50c0d7329fc24d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c8cca093b50c0d7329fc24d: start","job_status":"start","queue_duration_s":1.454321,"scheduling_latency_s":1.454337}1489gitlab # [ 270.923593] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3c8cca093b50c0d7329fc24d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3c8cca093b50c0d7329fc24d: done: 0.014447 sec","job_status":"done","queue_duration_s":1.454321,"scheduling_latency_s":1.454337,"redis_calls":13,"redis_duration_s":0.001508,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014447,"completed_at":"2026-09-13T13:04:06.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1490gitlab # [ 270.962220] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af8fc1220dc81a1d6a603516","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af8fc1220dc81a1d6a603516: start","job_status":"start","queue_duration_s":1.46986,"scheduling_latency_s":1.469877}1491gitlab # [ 270.974556] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af8fc1220dc81a1d6a603516","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af8fc1220dc81a1d6a603516: done: 0.014123 sec","job_status":"done","queue_duration_s":1.46986,"scheduling_latency_s":1.469877,"redis_calls":13,"redis_duration_s":0.001476,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014123,"completed_at":"2026-09-13T13:04:06.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1492gitlab # [ 271.013013] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2990f14738316882d170f9b1","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2990f14738316882d170f9b1: start","job_status":"start","queue_duration_s":1.485061,"scheduling_latency_s":1.485077}1493gitlab # [ 271.026109] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2990f14738316882d170f9b1","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2990f14738316882d170f9b1: done: 0.014233 sec","job_status":"done","queue_duration_s":1.485061,"scheduling_latency_s":1.485077,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014233,"completed_at":"2026-09-13T13:04:06.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1494gitlab # [ 271.064701] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4628e2870db5487796c39d5","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4628e2870db5487796c39d5: start","job_status":"start","queue_duration_s":1.500369,"scheduling_latency_s":1.500385}1495gitlab # [ 271.076896] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f4628e2870db5487796c39d5","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f4628e2870db5487796c39d5: done: 0.013956 sec","job_status":"done","queue_duration_s":1.500369,"scheduling_latency_s":1.500385,"redis_calls":13,"redis_duration_s":0.001403,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013956,"completed_at":"2026-09-13T13:04:06.612Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1496gitlab # [ 271.115246] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95d8a1ea48a4e6c7250bd9cc","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95d8a1ea48a4e6c7250bd9cc: start","job_status":"start","queue_duration_s":1.515429,"scheduling_latency_s":1.515446}1497gitlab # [ 271.127485] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.628Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"95d8a1ea48a4e6c7250bd9cc","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-95d8a1ea48a4e6c7250bd9cc: done: 0.014384 sec","job_status":"done","queue_duration_s":1.515429,"scheduling_latency_s":1.515446,"redis_calls":13,"redis_duration_s":0.001407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014384,"completed_at":"2026-09-13T13:04:06.628Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1498gitlab # [ 271.165768] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc1eb22089eee4936df6760d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc1eb22089eee4936df6760d: start","job_status":"start","queue_duration_s":1.530938,"scheduling_latency_s":1.530955}1499gitlab # [ 271.178227] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc1eb22089eee4936df6760d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc1eb22089eee4936df6760d: done: 0.013967 sec","job_status":"done","queue_duration_s":1.530938,"scheduling_latency_s":1.530955,"redis_calls":13,"redis_duration_s":0.0013909999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013967,"completed_at":"2026-09-13T13:04:06.643Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1500gitlab # [ 271.217020] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a996c4ab6ea8d17199fb29f","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a996c4ab6ea8d17199fb29f: start","job_status":"start","queue_duration_s":1.545958,"scheduling_latency_s":1.545975}1501gitlab # [ 271.229712] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a996c4ab6ea8d17199fb29f","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a996c4ab6ea8d17199fb29f: done: 0.014401 sec","job_status":"done","queue_duration_s":1.545958,"scheduling_latency_s":1.545975,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014401,"completed_at":"2026-09-13T13:04:06.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1502gitlab # [ 271.268711] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"248394c3328b13a92f7d9752","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-248394c3328b13a92f7d9752: start","job_status":"start","queue_duration_s":1.561436,"scheduling_latency_s":1.561453}1503gitlab # [ 271.280752] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"248394c3328b13a92f7d9752","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-248394c3328b13a92f7d9752: done: 0.014091 sec","job_status":"done","queue_duration_s":1.561436,"scheduling_latency_s":1.561453,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014091,"completed_at":"2026-09-13T13:04:06.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1504gitlab # [ 271.318585] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5026c6374b15e23e4969d1e0","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5026c6374b15e23e4969d1e0: start","job_status":"start","queue_duration_s":1.576604,"scheduling_latency_s":1.576621}1505gitlab # [ 271.330577] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5026c6374b15e23e4969d1e0","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5026c6374b15e23e4969d1e0: done: 0.014049 sec","job_status":"done","queue_duration_s":1.576604,"scheduling_latency_s":1.576621,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014049,"completed_at":"2026-09-13T13:04:06.689Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1506gitlab # [ 271.368875] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d206b712585da5d966918745","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d206b712585da5d966918745: start","job_status":"start","queue_duration_s":1.591736,"scheduling_latency_s":1.591753}1507gitlab # [ 271.380814] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d206b712585da5d966918745","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d206b712585da5d966918745: done: 0.014163 sec","job_status":"done","queue_duration_s":1.591736,"scheduling_latency_s":1.591753,"redis_calls":13,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009762,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014163,"completed_at":"2026-09-13T13:04:06.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1508gitlab # [ 271.418678] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f32e181bb111340826fc3658","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f32e181bb111340826fc3658: start","job_status":"start","queue_duration_s":1.607019,"scheduling_latency_s":1.607036}1509gitlab # [ 271.431054] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.719Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f32e181bb111340826fc3658","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f32e181bb111340826fc3658: done: 0.014418 sec","job_status":"done","queue_duration_s":1.607019,"scheduling_latency_s":1.607036,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014418,"completed_at":"2026-09-13T13:04:06.719Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1510gitlab # [ 271.470069] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04ef5fd7ee440d1698dc1676","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04ef5fd7ee440d1698dc1676: start","job_status":"start","queue_duration_s":1.622521,"scheduling_latency_s":1.622539}1511gitlab # [ 271.482148] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.735Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04ef5fd7ee440d1698dc1676","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04ef5fd7ee440d1698dc1676: done: 0.014095 sec","job_status":"done","queue_duration_s":1.622521,"scheduling_latency_s":1.622539,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009779,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014095,"completed_at":"2026-09-13T13:04:06.735Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1512gitlab # [ 271.520236] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eb71a3b50ddb597d4c36641","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eb71a3b50ddb597d4c36641: start","job_status":"start","queue_duration_s":1.637711,"scheduling_latency_s":1.637726}1513gitlab # [ 271.532268] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9eb71a3b50ddb597d4c36641","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9eb71a3b50ddb597d4c36641: done: 0.014066 sec","job_status":"done","queue_duration_s":1.637711,"scheduling_latency_s":1.637726,"redis_calls":13,"redis_duration_s":0.001397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009703,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014066,"completed_at":"2026-09-13T13:04:06.750Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1514gitlab # [ 271.570165] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"794759fa3825201c7b519cff","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-794759fa3825201c7b519cff: start","job_status":"start","queue_duration_s":1.652853,"scheduling_latency_s":1.65287}1515gitlab # [ 271.582429] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"794759fa3825201c7b519cff","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-794759fa3825201c7b519cff: done: 0.014258 sec","job_status":"done","queue_duration_s":1.652853,"scheduling_latency_s":1.65287,"redis_calls":13,"redis_duration_s":0.001434,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009891,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014258,"completed_at":"2026-09-13T13:04:06.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1516gitlab # [ 271.620567] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a20ff0bb319d02eda5cdc5b4","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a20ff0bb319d02eda5cdc5b4: start","job_status":"start","queue_duration_s":1.668191,"scheduling_latency_s":1.668209}1517gitlab # [ 271.632549] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a20ff0bb319d02eda5cdc5b4","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a20ff0bb319d02eda5cdc5b4: done: 0.014166 sec","job_status":"done","queue_duration_s":1.668191,"scheduling_latency_s":1.668209,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009843,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014166,"completed_at":"2026-09-13T13:04:06.780Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1518gitlab # [ 271.671769] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.782Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1eef04ad3733beb7de305772","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1eef04ad3733beb7de305772: start","job_status":"start","queue_duration_s":1.683605,"scheduling_latency_s":1.683622}1519gitlab # [ 271.684146] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1eef04ad3733beb7de305772","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1eef04ad3733beb7de305772: done: 0.013965 sec","job_status":"done","queue_duration_s":1.683605,"scheduling_latency_s":1.683622,"redis_calls":13,"redis_duration_s":0.001379,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013965,"completed_at":"2026-09-13T13:04:06.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1520gitlab # [ 271.722809] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d224003553b48e55612eb2b","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d224003553b48e55612eb2b: start","job_status":"start","queue_duration_s":1.698653,"scheduling_latency_s":1.698669}1521gitlab # [ 271.735255] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.810Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d224003553b48e55612eb2b","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d224003553b48e55612eb2b: done: 0.013889 sec","job_status":"done","queue_duration_s":1.698653,"scheduling_latency_s":1.698669,"redis_calls":13,"redis_duration_s":0.0013939999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000892,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013889,"completed_at":"2026-09-13T13:04:06.810Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1522gitlab # [ 271.774521] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4a8ea371732077700c77ad4","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4a8ea371732077700c77ad4: start","job_status":"start","queue_duration_s":1.713633,"scheduling_latency_s":1.71365}1523gitlab # [ 271.787032] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4a8ea371732077700c77ad4","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4a8ea371732077700c77ad4: done: 0.014503 sec","job_status":"done","queue_duration_s":1.713633,"scheduling_latency_s":1.71365,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014503,"completed_at":"2026-09-13T13:04:06.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1524gitlab # [ 271.826362] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.827Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211210c42576252bb7064631","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211210c42576252bb7064631: start","job_status":"start","queue_duration_s":1.729222,"scheduling_latency_s":1.729238}1525gitlab # [ 271.838854] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"211210c42576252bb7064631","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-211210c42576252bb7064631: done: 0.014005 sec","job_status":"done","queue_duration_s":1.729222,"scheduling_latency_s":1.729238,"redis_calls":13,"redis_duration_s":0.001407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014005,"completed_at":"2026-09-13T13:04:06.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1526gitlab # [ 271.879073] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a390eda7a7ebc1a7b84f68e","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a390eda7a7ebc1a7b84f68e: start","job_status":"start","queue_duration_s":1.744249,"scheduling_latency_s":1.744266}1527gitlab # [ 271.891369] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.857Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a390eda7a7ebc1a7b84f68e","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a390eda7a7ebc1a7b84f68e: done: 0.014427 sec","job_status":"done","queue_duration_s":1.744249,"scheduling_latency_s":1.744266,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009873,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014427,"completed_at":"2026-09-13T13:04:06.857Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1528gitlab # [ 271.930814] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d54f510a72a021ee5544162a","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d54f510a72a021ee5544162a: start","job_status":"start","queue_duration_s":1.759848,"scheduling_latency_s":1.759867}1529gitlab # [ 271.943467] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d54f510a72a021ee5544162a","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d54f510a72a021ee5544162a: done: 0.015114 sec","job_status":"done","queue_duration_s":1.759848,"scheduling_latency_s":1.759867,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015114,"completed_at":"2026-09-13T13:04:06.873Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1530gitlab # [ 271.982697] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76cbe242c8655db55177430a","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76cbe242c8655db55177430a: start","job_status":"start","queue_duration_s":1.776095,"scheduling_latency_s":1.776113}1531gitlab # [ 271.994988] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.888Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76cbe242c8655db55177430a","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76cbe242c8655db55177430a: done: 0.014244 sec","job_status":"done","queue_duration_s":1.776095,"scheduling_latency_s":1.776113,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014244,"completed_at":"2026-09-13T13:04:06.888Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1532gitlab # [ 272.034427] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a215181fa4d1a0422341667d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a215181fa4d1a0422341667d: start","job_status":"start","queue_duration_s":1.791413,"scheduling_latency_s":1.79143}1533gitlab # [ 272.046920] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.903Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a215181fa4d1a0422341667d","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a215181fa4d1a0422341667d: done: 0.013974 sec","job_status":"done","queue_duration_s":1.791413,"scheduling_latency_s":1.79143,"redis_calls":13,"redis_duration_s":0.0013930000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013974,"completed_at":"2026-09-13T13:04:06.903Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1534gitlab # [ 272.087580] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"394a2909f20c45704a240a25","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-394a2909f20c45704a240a25: start","job_status":"start","queue_duration_s":1.806466,"scheduling_latency_s":1.806483}1535gitlab # [ 272.100383] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.918Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"394a2909f20c45704a240a25","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-394a2909f20c45704a240a25: done: 0.014066 sec","job_status":"done","queue_duration_s":1.806466,"scheduling_latency_s":1.806483,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009752,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014066,"completed_at":"2026-09-13T13:04:06.918Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1536gitlab # [ 272.139136] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc06a38793d3d5cca5a24a11","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc06a38793d3d5cca5a24a11: start","job_status":"start","queue_duration_s":1.821825,"scheduling_latency_s":1.821843}1537gitlab # [ 272.151559] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc06a38793d3d5cca5a24a11","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc06a38793d3d5cca5a24a11: done: 0.014084 sec","job_status":"done","queue_duration_s":1.821825,"scheduling_latency_s":1.821843,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014084,"completed_at":"2026-09-13T13:04:06.934Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538gitlab # [ 272.190343] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.935Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68a450ffe908a2326c413dd1","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68a450ffe908a2326c413dd1: start","job_status":"start","queue_duration_s":1.83695,"scheduling_latency_s":1.836966}1539gitlab # [ 272.202621] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68a450ffe908a2326c413dd1","created_at":"2026-09-13T13:04:04.936Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-978bcb9d23f44eb4","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"5432170235abdf1fa332f55367ddf323","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.098Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68a450ffe908a2326c413dd1: done: 0.014018 sec","job_status":"done","queue_duration_s":1.83695,"scheduling_latency_s":1.836966,"redis_calls":13,"redis_duration_s":0.001399,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014018,"completed_at":"2026-09-13T13:04:06.949Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1540gitlab # [ 272.241289] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.950Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c87c97d0e19615f6ccce9a1a","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c87c97d0e19615f6ccce9a1a: start","job_status":"start","queue_duration_s":1.414424,"scheduling_latency_s":1.414441}1541gitlab # [ 272.253565] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.962Z","message":"No existing merge request to be cleaned up."}1542gitlab # [ 272.255248] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.971Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c87c97d0e19615f6ccce9a1a","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c87c97d0e19615f6ccce9a1a: done: 0.020747 sec","job_status":"done","queue_duration_s":1.414424,"scheduling_latency_s":1.414441,"redis_calls":10,"redis_duration_s":0.001308,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000739,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013694,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020747,"completed_at":"2026-09-13T13:04:06.971Z","load_balancing_strategy":"primary","db_duration_s":0.00374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 272.294602] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.972Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f349e9eced3062e610eea262","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f349e9eced3062e610eea262: start","job_status":"start","queue_duration_s":1.436371,"scheduling_latency_s":1.436388}1544gitlab # [ 272.306888] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.983Z","message":"No existing merge request to be cleaned up."}1545gitlab # [ 272.308542] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.991Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"f349e9eced3062e610eea262","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-f349e9eced3062e610eea262: done: 0.018856 sec","job_status":"done","queue_duration_s":1.436371,"scheduling_latency_s":1.436388,"redis_calls":10,"redis_duration_s":0.001289,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000723,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018856,"completed_at":"2026-09-13T13:04:06.991Z","load_balancing_strategy":"primary","db_duration_s":0.002669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1546gitlab # [ 272.346568] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:06.992Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9bafd26e916f0dca0c257f01","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9bafd26e916f0dca0c257f01: start","job_status":"start","queue_duration_s":1.456315,"scheduling_latency_s":1.456331}1547gitlab # [ 272.358639] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:07.002Z","message":"No existing merge request to be cleaned up."}1548gitlab # [ 272.360310] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:07.010Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9bafd26e916f0dca0c257f01","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9bafd26e916f0dca0c257f01: done: 0.018282 sec","job_status":"done","queue_duration_s":1.456315,"scheduling_latency_s":1.456331,"redis_calls":10,"redis_duration_s":0.0013,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00073,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018282,"completed_at":"2026-09-13T13:04:07.010Z","load_balancing_strategy":"primary","db_duration_s":0.002236,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 272.398626] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:07.011Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1711db8c85bb09271b7de1ca","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1711db8c85bb09271b7de1ca: start","job_status":"start","queue_duration_s":1.475681,"scheduling_latency_s":1.475698}1550gitlab # [ 272.410843] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:07.022Z","message":"No existing merge request to be cleaned up."}1551gitlab # [ 272.412499] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:07.030Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"1711db8c85bb09271b7de1ca","created_at":"2026-09-13T13:04:05.526Z","trace_propagation_headers":{"sentry-trace":"0f0205a62c2449e38d637b954c5790b2-b04b5690eb6845de","baggage":"sentry-trace_id=0f0205a62c2449e38d637b954c5790b2,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"7434463e7c921305c80cacf21c3eb61d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:04:05.536Z","job_size_bytes":2,"pid":2027,"sidekiq_tid":"mao3","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-1711db8c85bb09271b7de1ca: done: 0.018452 sec","job_status":"done","queue_duration_s":1.475681,"scheduling_latency_s":1.475698,"redis_calls":10,"redis_duration_s":0.001292,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018452,"completed_at":"2026-09-13T13:04:07.030Z","load_balancing_strategy":"primary","db_duration_s":0.002269,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1552gitlab # [ 272.943239] setup-gitlab-start[2959]: W, [2026-09-13T13:04:09.831062 #2959] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1553gitlab # [ 309.181948] redis[794]: 10 changes in 300 seconds. Saving...1554gitlab # [ 309.188584] redis[794]: Background saving started by pid 32931555gitlab # [ 309.192826] redis[3293]: BGSAVE done, 453 keys saved, 0 keys skipped, 117567 bytes written.1556gitlab # [ 309.199673] redis[3293]: DB saved on disk1557gitlab # [ 309.202088] redis[3293]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1558gitlab # [ 309.283088] redis[794]: Background saving terminated with success1559gitlab # [ 310.696975] postgres[861]: [861] LOG: checkpoint starting: time1560gitlab # [ 311.952659] gitaly[919]: time="2026-09-13T13:04:48.838Z" level=info msg="burden monitor stats" pid=9191561gitlab # [ 315.825352] postgres[3294]: [3294] ERROR: relation "table_without_models" does not exist1562gitlab # [ 315.825699] postgres[3294]: [3294] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1563gitlab # [ 322.573851] runuser[2958]: pam_unix(runuser:session): session closed for user gitlab1564gitlab # [ 322.575700] setup-gitlab-start[2925]: + systemctl restart gitlab-sidekiq.service1565gitlab # [ 322.615243] systemd[1]: Stopping gitlab-sidekiq.service...1566gitlab # [ 322.653063] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:59.540Z","message":"Shutting down"}1567gitlab # [ 322.657416] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:59.541Z","message":"Scheduler exiting..."}1568gitlab # [ 322.661626] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:59.541Z","message":"Terminating quiet threads for default capsule"}1569gitlab # [ 322.666770] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:04:59.542Z","message":"Scheduler exiting..."}1570gitlab # [ 323.154764] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:05:00.043Z","message":"Pausing to allow jobs to finish..."}1571gitlab # [ 325.659355] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:05:02.546Z","message":"Bye!"}1572gitlab # [ 325.701280] sidekiq-cluster[2027]: {"severity":"INFO","time":"2026-09-13T13:05:02.589Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2027,"worker_id":"sidekiq_0","memwd_rss_bytes":967548928,"retry":0}1573gitlab # [ 326.640273] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1574gitlab # [ 326.640711] systemd[1]: Stopped gitlab-sidekiq.service.1575gitlab # [ 326.640930] systemd[1]: gitlab-sidekiq.service: Consumed 1min 58.052s CPU time over 3min 23.269s wall clock time, 951.6M memory peak, 4.4M read from disk.1576gitlab # [ 326.654311] systemd[1]: Started gitlab-sidekiq.service.1577gitlab # [ 326.660843] setup-gitlab-start[2925]: + mkdir -p /var/lib/secrets1578gitlab # [ 326.667154] setup-gitlab-start[2925]: + echo nixbot-test-token-of-20-chars1579gitlab # [ 326.670281] setup-gitlab-start[2925]: + chmod 644 /var/lib/secrets/gitlab-token1580gitlab # [ 326.674499] setup-gitlab-start[2925]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1581gitlab # [ 327.324913] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:04 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 6011582gitlab # [ 328.407340] gitaly[919]: time="2026-09-13T13:05:05.295Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.294" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 328.407845] gitaly[919]: time="2026-09-13T13:05:05.295Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.294" grpc.time_ms=0.766 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 328.609823] gitaly[919]: time="2026-09-13T13:05:05.498Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.207" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.498" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 328.640836] gitaly[919]: time="2026-09-13T13:05:05.499Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.207" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.498" grpc.time_ms=0.544 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586gitlab # [ 328.674798] gitaly[919]: time="2026-09-13T13:05:05.563Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN diskcache=b97818a6-7639-4d94-bae9-c5822e99785d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T13:05:15.217" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.508" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1587gitlab # [ 328.701451] gitaly[919]: time="2026-09-13T13:05:05.564Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=26 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=32 command.real_time_ms=32 command.spawn_token_fork_ms=8 command.system_time_ms=19 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T13:05:15.217" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.508" grpc.time_ms=55.101 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1588gitlab # [ 328.726072] gitaly[919]: time="2026-09-13T13:05:05.578Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.287" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.578" pid=919 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1589gitlab # [ 328.743207] gitaly[919]: time="2026-09-13T13:05:05.579Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.287" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.578" grpc.time_ms=0.967 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1590gitlab # [ 328.759671] gitaly[919]: time="2026-09-13T13:05:05.600Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:05:15.293" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.583" grpc.time_ms=16.582 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1591gitlab # [ 328.797226] gitaly[919]: time="2026-09-13T13:05:05.686Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.686" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1592gitlab # [ 328.814548] gitaly[919]: time="2026-09-13T13:05:05.686Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.686" grpc.time_ms=0.281 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593gitlab # [ 328.842752] gitaly[919]: time="2026-09-13T13:05:05.731Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN diskcache=a9aeb6f3-3185-4fcf-9c21-57378226b171 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T13:05:15.008" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.688" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594gitlab # [ 328.858269] gitaly[919]: time="2026-09-13T13:05:05.732Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=26 command.spawn_token_fork_ms=5 command.system_time_ms=15 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T13:05:15.008" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.688" grpc.time_ms=43.617 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595gitlab # [ 328.877000] gitaly[919]: time="2026-09-13T13:05:05.738Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.057" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.737" pid=919 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1596gitlab # [ 328.889694] gitaly[919]: time="2026-09-13T13:05:05.738Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.057" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.737" grpc.time_ms=0.526 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1597gitlab # [ 328.901887] gitaly[919]: time="2026-09-13T13:05:05.759Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:05:15.062" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.742" grpc.time_ms=16.094 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598gitlab # [ 328.916987] gitaly[919]: time="2026-09-13T13:05:05.794Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:05:15.113" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.793" grpc.time_ms=0.275 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1599gitlab # [ 328.929122] gitaly[919]: time="2026-09-13T13:05:05.803Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T13:05:15.121" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:05.802" grpc.time_ms=0.663 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1600gitlab # [ 329.732278] gitaly[919]: time="2026-09-13T13:05:06.620Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDY37KBWN4SBRX5XT0AGBN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:05:16.000" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:06.602" grpc.time_ms=17.65 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1601gitlab # [ 330.121007] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:07 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 27641602gitlab # [ 330.127107] setup-gitlab-start[3383]: {"id":1,"message":null,"error":null}1603gitlab # [ 331.585512] sidekiq-cluster[3365]: {"severity":"INFO","time":"2026-09-13T13:05:08.445Z","message":"Starting cluster with 1 processes"}1604gitlab # [ 335.908845] gitaly[919]: time="2026-09-13T13:05:12.797Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYAVC2S03JMK7X8RF8RZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:05:22.008" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:12.779" grpc.time_ms=17.65 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1605gitlab # [ 338.572484] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:15 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 34151606gitlab # [ 338.576984] setup-gitlab-start[2925]: + mkdir -p /tmp/test-flake1607gitlab # [ 338.581666] setup-gitlab-start[2925]: + cd /tmp/test-flake1608gitlab # [ 338.586427] setup-gitlab-start[2925]: + git init -b master1609gitlab # [ 338.609949] setup-gitlab-start[3481]: Initialized empty Git repository in /tmp/test-flake/.git/1610gitlab # [ 338.613717] setup-gitlab-start[2925]: + git config user.name test1611gitlab # [ 338.617503] setup-gitlab-start[2925]: + git config user.email test@example.com1612gitlab # [ 338.624784] setup-gitlab-start[2925]: + cat1613gitlab # [ 338.630382] setup-gitlab-start[2925]: + git add flake.nix1614gitlab # [ 338.639303] setup-gitlab-start[2925]: + git commit -m 'initial commit'1615gitlab # [ 338.669982] setup-gitlab-start[3486]: [master (root-commit) aa72696] initial commit1616gitlab # [ 338.674319] setup-gitlab-start[3486]: 1 file changed, 10 insertions(+)1617gitlab # [ 338.677339] setup-gitlab-start[3486]: create mode 100644 flake.nix1618gitlab # [ 338.680081] setup-gitlab-start[2925]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1619gitlab # [ 338.685260] setup-gitlab-start[2925]: + git push -u origin master1620gitlab # [ 338.806003] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:15 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 761621gitlab # [ 339.017825] gitaly[919]: time="2026-09-13T13:05:15.906Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEDPTWEZQY12N4VMYMWF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:15.889" grpc.time_ms=16.996 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1622gitlab # [ 339.039114] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:15 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 2051623gitlab # [ 339.451373] gitaly[919]: time="2026-09-13T13:05:16.340Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:05:26.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:16.323" grpc.time_ms=16.693 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1624gitlab # [ 339.535675] gitaly[919]: time="2026-09-13T13:05:16.424Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T13:05:46.006" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T13:05:16.407" grpc.time_ms=17.143 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1625gitlab # [ 339.596266] gitaly[919]: time="2026-09-13T13:05:16.485Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T duration_ms=272ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/allowed"1626gitlab # [ 339.671773] gitaly[919]: time="2026-09-13T13:05:16.560Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T duration_ms=65ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/pre_receive"1627gitlab # [ 339.684768] gitaly[919]: time="2026-09-13T13:05:16.561Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:05:16.210" grpc.time_ms=350.974 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1628gitlab # [ 339.767718] gitaly[919]: time="2026-09-13T13:05:16.656Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:05:16.655" grpc.time_ms=1.319 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1629gitlab # [ 339.827241] gitaly[919]: time="2026-09-13T13:05:16.716Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:05:16.712" grpc.time_ms=3.034 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1630gitlab # [ 339.891274] gitaly[919]: time="2026-09-13T13:05:16.780Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:05:16.778" grpc.time_ms=1.846 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1631gitlab # [ 340.095712] gitaly[919]: time="2026-09-13T13:05:16.983Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:05:26.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:16.966" grpc.time_ms=17.127 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1632gitlab # [ 340.147482] gitaly[919]: time="2026-09-13T13:05:17.036Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1434 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=4 command.system_time_ms=24 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T13:05:26.037" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:16.998" grpc.time_ms=37.814 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1633gitlab # [ 340.187674] gitaly[919]: time="2026-09-13T13:05:17.076Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T duration_ms=212ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/post_receive"1634gitlab # [ 340.196350] gitaly[919]: time="2026-09-13T13:05:17.077Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:05:16.862" grpc.time_ms=214.925 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1635gitlab # [ 340.216611] gitaly[919]: time="2026-09-13T13:05:17.092Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T diskcache=9014f4a2-4fef-4ea2-a742-23f4a08a1ef5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:16.105" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1636gitlab # [ 340.233860] setup-gitlab-start[3490]: To http://gitlab/root/test-flake.git1637gitlab # [ 340.235292] setup-gitlab-start[3490]: * [new branch] master -> master1638gitlab # [ 340.237670] setup-gitlab-start[3490]: branch 'master' set up to track 'origin/master'.1639gitlab # [ 340.239702] gitlab-workhorse[819]: time="2026-09-13T13:05:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521640gitlab # [ 340.242571] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:17 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 11641641gitlab # [ 340.245807] systemd[1]: Finished setup-gitlab.service.1642gitlab # [ 340.246960] gitaly[919]: time="2026-09-13T13:05:17.093Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=358 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=856064 command.minflt=13933 command.oublock=32 command.real_time_ms=976 command.spawn_token_fork_ms=2 command.system_time_ms=221 command.user_time_ms=137 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:16.105" grpc.time_ms=987.017 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1643gitlab # [ 340.267500] systemd[1]: Starting nixbot CI service...1644gitlab: (finished: waiting for unit setup-gitlab.service, in 340.94 seconds)1645gitlab: waiting for unit nixbot.service1646gitlab # [ 341.636328] nixbot[3565]: {"ts": "2026-09-13T13:05:18.524Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1647gitlab # [ 341.676191] nixbot[3565]: {"ts": "2026-09-13T13:05:18.565Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1648gitlab # [ 341.715351] nixbot[3565]: {"ts": "2026-09-13T13:05:18.604Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1649gitlab # [ 341.722105] nixbot[3565]: {"ts": "2026-09-13T13:05:18.610Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1650gitlab # [ 341.729103] nixbot[3565]: {"ts": "2026-09-13T13:05:18.616Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1651gitlab # [ 341.740640] nixbot[3565]: {"ts": "2026-09-13T13:05:18.629Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1652gitlab # [ 341.747619] nixbot[3565]: {"ts": "2026-09-13T13:05:18.636Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1653gitlab # [ 341.754310] nixbot[3565]: {"ts": "2026-09-13T13:05:18.639Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1654gitlab # [ 341.760876] nixbot[3565]: {"ts": "2026-09-13T13:05:18.649Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1655gitlab # [ 341.767438] nixbot[3565]: {"ts": "2026-09-13T13:05:18.652Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1656gitlab # [ 341.774410] nixbot[3565]: {"ts": "2026-09-13T13:05:18.660Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1657gitlab # [ 341.781569] nixbot[3565]: {"ts": "2026-09-13T13:05:18.670Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1658gitlab # [ 341.788269] nixbot[3565]: {"ts": "2026-09-13T13:05:18.675Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1659gitlab # [ 341.795029] nixbot[3565]: {"ts": "2026-09-13T13:05:18.679Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1660gitlab # [ 341.801574] nixbot[3565]: {"ts": "2026-09-13T13:05:18.685Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1661gitlab # [ 341.807720] nixbot[3565]: {"ts": "2026-09-13T13:05:18.687Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1662gitlab # [ 341.813894] nixbot[3565]: {"ts": "2026-09-13T13:05:18.690Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1663gitlab # [ 341.819966] nixbot[3565]: {"ts": "2026-09-13T13:05:18.692Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1664gitlab # [ 341.825381] nixbot[3565]: {"ts": "2026-09-13T13:05:18.699Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1665gitlab # [ 341.830976] nixbot[3565]: {"ts": "2026-09-13T13:05:18.702Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1666gitlab # [ 341.836668] nixbot[3565]: {"ts": "2026-09-13T13:05:18.704Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1667gitlab # [ 341.841704] nixbot[3565]: {"ts": "2026-09-13T13:05:18.712Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1668gitlab # [ 341.846776] nixbot[3565]: {"ts": "2026-09-13T13:05:18.714Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1669gitlab # [ 341.851463] nixbot[3565]: {"ts": "2026-09-13T13:05:18.717Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1670gitlab # [ 341.855909] nixbot[3565]: {"ts": "2026-09-13T13:05:18.719Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1671gitlab # [ 341.860464] nixbot[3565]: {"ts": "2026-09-13T13:05:18.723Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1672gitlab # [ 341.865217] nixbot[3565]: {"ts": "2026-09-13T13:05:18.725Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1673gitlab # [ 341.870885] nixbot[3565]: {"ts": "2026-09-13T13:05:18.727Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1674gitlab # [ 341.875262] nixbot[3565]: {"ts": "2026-09-13T13:05:18.729Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1675gitlab # [ 341.879622] nixbot[3565]: {"ts": "2026-09-13T13:05:18.744Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1676gitlab # [ 341.883694] nixbot[3565]: {"ts": "2026-09-13T13:05:18.750Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1677gitlab # [ 341.887833] nixbot[3565]: {"ts": "2026-09-13T13:05:18.753Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1678gitlab # [ 341.891920] nixbot[3565]: {"ts": "2026-09-13T13:05:18.754Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1679gitlab # [ 341.895853] nixbot[3565]: {"ts": "2026-09-13T13:05:18.760Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1680gitlab # [ 341.899639] nixbot[3565]: {"ts": "2026-09-13T13:05:18.762Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1681gitlab # [ 341.903113] nixbot[3565]: {"ts": "2026-09-13T13:05:18.769Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1682gitlab # [ 342.263746] nixbot[3565]: INFO: Started server process [3565]1683gitlab # [ 342.264814] nixbot[3565]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1684gitlab # [ 342.317331] nixbot[3565]: INFO: ::ffff:127.0.0.1:41088 - "GET /health HTTP/1.1" 200 OK1685gitlab # [ 342.328369] systemd[1]: Started nixbot CI service.1686gitlab # [ 342.330027] systemd[1]: Reached target Multi-User System.1687gitlab # [ 342.332474] systemd[1]: Startup finished in 759ms (kernel) + 4.075s (initrd) + 5min 37.494s (userspace) = 5min 42.329s.1688gitlab: (finished: waiting for unit nixbot.service, in 2.14 seconds)1689gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1690gitlab # [ 342.676184] nixbot[3565]: INFO: ::ffff:127.0.0.1:33152 - "GET /health HTTP/1.1" 200 OK1691gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.06 seconds)1692subtest: project discovered and webhook registered1693gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1694gitlab # [ 342.739171] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:19 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 4221695gitlab # [ 342.747952] nixbot[3565]: {"ts": "2026-09-13T13:05:19.629Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1696gitlab # [ 342.774283] nixbot[3565]: {"ts": "2026-09-13T13:05:19.662Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1697gitlab # [ 342.851206] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 1141698gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1699gitlab # [ 342.884420] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:19 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 921700gitlab # [ 342.891998] nixbot[3565]: {"ts": "2026-09-13T13:05:19.774Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1701gitlab # [ 342.899327] nixbot[3565]: {"ts": "2026-09-13T13:05:19.777Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1702gitlab # [ 343.036280] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1391703gitlab # [ 343.043988] nixbot[3565]: {"ts": "2026-09-13T13:05:19.922Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1704gitlab # [ 343.221537] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:20 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1691705gitlab # [ 343.223942] nixbot[3565]: {"ts": "2026-09-13T13:05:20.112Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1706gitlab # [ 343.341143] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 1031707gitlab # [ 343.343342] nixbot[3565]: {"ts": "2026-09-13T13:05:20.232Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1708gitlab # [ 343.422832] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:20 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 711709gitlab # [ 343.429583] nixbot[3565]: {"ts": "2026-09-13T13:05:20.313Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1710gitlab # [ 343.520078] gitaly[919]: time="2026-09-13T13:05:20.406Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYJY30DZ1WH9SEFG4MGYD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-13T13:05:50.078" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:20.379" grpc.time_ms=26.298 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711gitlab # [ 343.545056] gitaly[919]: time="2026-09-13T13:05:20.433Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYJY30DZ1WH9SEFG4MGYD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:05:50.031" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:05:20.432" grpc.time_ms=1.269 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 343.579232] gitaly[919]: time="2026-09-13T13:05:20.455Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYJY30DZ1WH9SEFG4MGYD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-13T13:05:30.120" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:05:20.440" grpc.time_ms=14.06 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1713gitlab # [ 343.604176] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 1631714gitlab # [ 343.607551] nixbot[3565]: {"ts": "2026-09-13T13:05:20.489Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1715gitlab # [ 343.746368] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1371716gitlab # [ 343.754999] nixbot[3565]: {"ts": "2026-09-13T13:05:20.636Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1717gitlab # [ 343.763092] nixbot[3565]: {"ts": "2026-09-13T13:05:20.647Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1718gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1719gitlab # [ 343.948201] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:20 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1121720gitlab # [ 343.952805] gitaly[919]: time="2026-09-13T13:05:20.838Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYKAMRCWT69YNHXPKT6JS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:20.821" grpc.time_ms=14.651 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1721gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds)1722gitlab # [ 344.024452] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:20 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 1161723(finished: subtest: project discovered and webhook registered, in 1.35 seconds)1724subtest: push delivers the webhook1725gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1726gitlab # [ 344.165030] gitaly[919]: time="2026-09-13T13:05:21.052Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKEB4A43K80S6E2STHJ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.033" pid=919 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1727gitlab # [ 344.195459] gitaly[919]: time="2026-09-13T13:05:21.053Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKEB4A43K80S6E2STHJ6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.033" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1728gitlab # [ 344.227062] gitlab-workhorse[819]: time="2026-09-13T13:05:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131729gitlab # [ 344.230795] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 2111730gitlab # [ 344.234322] gitaly[919]: time="2026-09-13T13:05:21.053Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKEB4A43K80S6E2STHJ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.033" grpc.time_ms=20.022 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1731gitlab # [ 344.271804] gitaly[919]: time="2026-09-13T13:05:21.160Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKN41BT8Z5XFV1HDK86B grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.139" pid=919 remote_ip="2001:db8:1::1" request_sha=9bc082ca8d89794312821cadd94c5baec6b12569 response_bytes=223 span.kind=server system=grpc1732gitlab # [ 344.288665] gitaly[919]: time="2026-09-13T13:05:21.160Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKN41BT8Z5XFV1HDK86B grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.139" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1733gitlab # [ 344.309478] gitlab-workhorse[819]: time="2026-09-13T13:05:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231734gitlab # [ 344.312188] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1011735gitlab # [ 344.314817] gitaly[919]: time="2026-09-13T13:05:21.161Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKN41BT8Z5XFV1HDK86B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.139" grpc.time_ms=21.837 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1736gitlab # [ 344.435422] gitaly[919]: time="2026-09-13T13:05:21.324Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=7 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYKT5Z20EY7HQFPXFQRX3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.302" grpc.time_ms=22.105 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1737gitlab # [ 344.472701] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1041738gitlab # [ 344.729749] gitaly[919]: time="2026-09-13T13:05:21.617Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKXG2FWE2REPENYKHVQ7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.598" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1739gitlab # [ 344.763619] gitaly[919]: time="2026-09-13T13:05:21.617Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKXG2FWE2REPENYKHVQ7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.598" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1740gitlab # [ 344.791956] gitlab-workhorse[819]: time="2026-09-13T13:05:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671741gitlab # [ 344.795256] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 2901742gitlab # [ 344.798605] gitaly[919]: time="2026-09-13T13:05:21.618Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYKXG2FWE2REPENYKHVQ7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.598" grpc.time_ms=19.305 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1743gitlab # [ 344.869324] gitaly[919]: time="2026-09-13T13:05:21.758Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYM8RRFKRQCBSQSC0CTZE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.741" grpc.time_ms=16.898 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1744gitlab # [ 344.907104] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:21 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 701745gitlab # [ 344.950037] gitaly[919]: time="2026-09-13T13:05:21.838Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMB0MHBQ5PW45Q9RQ0ZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.814" pid=919 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1746gitlab # [ 344.983248] gitaly[919]: time="2026-09-13T13:05:21.839Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMB0MHBQ5PW45Q9RQ0ZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.814" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1747gitlab # [ 345.019930] gitlab-workhorse[819]: time="2026-09-13T13:05:21Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091748gitlab # [ 345.023869] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:21 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 781749gitlab # [ 345.027613] gitaly[919]: time="2026-09-13T13:05:21.839Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMB0MHBQ5PW45Q9RQ0ZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:21.814" grpc.time_ms=24.821 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1750gitlab # [ 345.071126] gitaly[919]: time="2026-09-13T13:05:21.960Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:05:51.558" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:05:21.959" grpc.time_ms=0.783 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1751gitlab # [ 345.157761] gitaly[919]: time="2026-09-13T13:05:22.046Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T13:05:52.031" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:22.032" grpc.time_ms=14.284 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1752gitlab # [ 345.191072] gitaly[919]: time="2026-09-13T13:05:22.080Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:05:32.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:22.065" grpc.time_ms=14.229 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1753gitlab # [ 345.217247] gitaly[919]: time="2026-09-13T13:05:22.106Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:05:32.029" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:22.089" grpc.time_ms=16.461 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754gitlab # [ 345.353037] gitaly[919]: time="2026-09-13T13:05:22.241Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T13:05:32.180" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:22.241" grpc.time_ms=0.644 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1755gitlab # [ 345.383056] gitaly[919]: time="2026-09-13T13:05:22.271Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T13:05:32.196" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:22.257" grpc.time_ms=14.162 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1756gitlab # [ 345.475918] gitaly[919]: time="2026-09-13T13:05:22.364Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:05:22.364" grpc.time_ms=0.724 pid=919 span.kind=server system=grpc1757gitlab # [ 345.529821] gitaly[919]: time="2026-09-13T13:05:22.418Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:05:22.417" grpc.time_ms=0.712 pid=919 span.kind=server system=grpc1758gitlab # [ 345.554222] gitaly[919]: time="2026-09-13T13:05:22.430Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 diskcache=c8532b5d-9678-467c-bd4a-513da8fdae9a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T13:05:32.234" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:22.295" pid=919 ref=refs/keep-around/aa726960fcb15de306f9a352e35de564cfe41bbb remote_ip="2001:db8:1::1" revision=aa726960fcb15de306f9a352e35de564cfe41bbb span.kind=server system=grpc username=root1759gitlab # [ 345.576411] gitaly[919]: time="2026-09-13T13:05:22.442Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=138 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5489 command.oublock=8 command.real_time_ms=128 command.spawn_token_fork_ms=1 command.system_time_ms=90 command.user_time_ms=48 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T13:05:32.234" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:05:22.295" grpc.time_ms=147.129 pid=919 ref=refs/keep-around/aa726960fcb15de306f9a352e35de564cfe41bbb remote_ip="2001:db8:1::1" revision=aa726960fcb15de306f9a352e35de564cfe41bbb span.kind=server system=grpc username=root1760gitlab # [ 345.602765] gitaly[919]: time="2026-09-13T13:05:22.450Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:05:52.048" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:05:22.448" grpc.time_ms=0.961 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761gitlab # [ 345.863484] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:22 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 8651762gitlab # [ 345.872869] nixbot[3565]: {"ts": "2026-09-13T13:05:22.753Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb \"HTTP/1.1 201 Created\""}1763gitlab # [ 345.882300] nixbot[3565]: {"ts": "2026-09-13T13:05:22.770Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-aa726960fcb1-0"}1764gitlab # [ 346.161131] nixbot[3565]: {"ts": "2026-09-13T13:05:23.049Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-aa726960fcb1-0", "argv0": "bwrap"}1765gitlab # [ 346.349073] systemd[1]: Started Nix Daemon.1766gitlab # [ 346.464058] nix-daemon[3738]: accepted connection from pid 3729, user nixbot1767gitlab # [ 350.898746] nix-daemon[3738]: accepted connection from pid 3778, user nixbot1768gitlab # [ 350.922532] nix-daemon[3738]: accepted connection from pid 3729, user nixbot1769gitlab # [ 355.552342] nix-daemon[3738]: accepted connection from pid 3729, user nixbot1770gitlab # [ 355.755354] gitaly[919]: time="2026-09-13T13:05:32.643Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYYVKB3YZDB291F0S8D1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:06:02.041" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:05:32.642" grpc.time_ms=1.086 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1771gitlab # [ 356.276346] gitaly[919]: time="2026-09-13T13:05:33.165Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYYVKB3YZDB291F0S8D1C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:05:43.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:33.147" grpc.time_ms=17.521 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772gitlab # [ 356.541666] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 8971773gitlab # [ 356.548492] nixbot[3565]: {"ts": "2026-09-13T13:05:33.432Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb \"HTTP/1.1 201 Created\""}1774gitlab # [ 356.671237] gitaly[919]: time="2026-09-13T13:05:33.560Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYZR10M3C464KW4SN7WJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:06:03.057" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:05:33.558" grpc.time_ms=1.326 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1775gitlab # [ 356.760355] gitaly[919]: time="2026-09-13T13:05:33.647Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=3 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYZR10M3C464KW4SN7WJC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:05:43.025" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:33.626" grpc.time_ms=21.295 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1776gitlab # [ 356.882997] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3291777gitlab # [ 356.891127] nixbot[3565]: {"ts": "2026-09-13T13:05:33.773Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb \"HTTP/1.1 201 Created\""}1778gitlab # [ 357.020461] gitaly[919]: time="2026-09-13T13:05:33.909Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDZ032A0M9D9Y7JS2R2PY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:06:03.506" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:05:33.907" grpc.time_ms=1.343 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779gitlab # [ 357.100124] gitaly[919]: time="2026-09-13T13:05:33.988Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M2DDZ032A0M9D9Y7JS2R2PY0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:05:43.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:05:33.973" grpc.time_ms=15.088 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1780gitlab # [ 357.246992] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:34 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 3411781gitlab # [ 357.255183] nixbot[3565]: {"ts": "2026-09-13T13:05:34.137Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb \"HTTP/1.1 201 Created\""}1782gitlab # [ 357.909061] gitaly[919]: time="2026-09-13T13:05:34.796Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DDZ0YFMZM138JHQC28ZM4Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:34.780" grpc.time_ms=16.502 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1783gitlab # [ 357.911294] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:34 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1251784gitlab # [ 358.035708] gitaly[919]: time="2026-09-13T13:05:34.924Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDZ13DMYVC5EC1TXPX3ZXP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:34.907" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1785gitlab # [ 358.051505] gitaly[919]: time="2026-09-13T13:05:34.924Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDZ13DMYVC5EC1TXPX3ZXP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:34.907" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1786gitlab # [ 358.069441] gitlab-workhorse[819]: time="2026-09-13T13:05:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671787gitlab # [ 358.071706] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:34 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 961788gitlab # [ 358.074055] gitaly[919]: time="2026-09-13T13:05:34.925Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDZ13DMYVC5EC1TXPX3ZXP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:34.907" grpc.time_ms=17.841 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1789gitlab # [ 358.218992] gitaly[919]: time="2026-09-13T13:05:35.107Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M2DDZ18RG6N11Q8RH75TN3HM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:35.095" grpc.time_ms=12.242 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1790gitlab # [ 358.236339] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:35 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1071791gitlab # [ 358.368362] gitaly[919]: time="2026-09-13T13:05:35.257Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDZ1CRQA0Z6HPWRM1W5BB0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:35.237" pid=919 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1792gitlab # [ 358.398956] gitaly[919]: time="2026-09-13T13:05:35.257Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDZ1CRQA0Z6HPWRM1W5BB0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:35.237" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1793gitlab # [ 358.423355] gitlab-workhorse[819]: time="2026-09-13T13:05:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671794gitlab # [ 358.426370] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:05:35 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1291795gitlab # [ 358.429258] gitaly[919]: time="2026-09-13T13:05:35.258Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDZ1CRQA0Z6HPWRM1W5BB0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:05:35.237" grpc.time_ms=19.96 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1796gitlab # [ 372.420552] sidekiq-cluster[3435]: 2026-09-13T13:05:49.308Z pid=3435 tid=ypv INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3435, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>424570880}1797gitlab # [ 372.781440] sidekiq-cluster[3435]: W, [2026-09-13T13:05:49.670305 #3435] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1798gitlab # [ 372.856378] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:05:49.745Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1799gitlab # [ 375.856246] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:05:52.744Z","message":"GitLab reliable fetch activated!"}1800gitlab # [ 375.859496] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:05:52.747Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1801gitlab # [ 411.838973] postgres[4272]: [4272] ERROR: relation "table_without_models" does not exist1802gitlab # [ 411.839573] postgres[4272]: [4272] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1803gitlab # [ 416.726016] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:33.614Z","message":"Booted Rails 7.2.3.1 application in production environment"}1804gitlab # [ 416.728934] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:33.617Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1805gitlab # [ 416.731421] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:33.620Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1806gitlab # [ 416.736237] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:33.620Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1807gitlab # [ 416.738214] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:33.623Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1808gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 73.22 seconds)1809gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1810gitlab # [ 417.478510] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:34 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1381811gitlab # [ 417.697519] gitaly[919]: time="2026-09-13T13:06:34.585Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0V7ZC8NYNCB0P151KGYZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:34.567" grpc.time_ms=17.212 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1812gitlab # [ 417.738499] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:34 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 200 "" "git/2.55.0" 2171813gitlab # Enumerating objects: 5, done.1814gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1815gitlab # Delta compression using up to 4 threads1816gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1817gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 91.00 KiB/s, done.1818gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1819gitlab # [ 418.348640] gitaly[919]: time="2026-09-13T13:06:35.237Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T13:07:05.015" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T13:06:35.215" grpc.time_ms=21.436 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1820gitlab # [ 418.396034] gitaly[919]: time="2026-09-13T13:06:35.284Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2DE0VH64JMW9MMND5PDBXRD duration_ms=313ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/allowed"1821gitlab # [ 418.455689] gitaly[919]: time="2026-09-13T13:06:35.344Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2DE0VH64JMW9MMND5PDBXRD duration_ms=58ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/pre_receive"1822gitlab # [ 418.464659] gitaly[919]: time="2026-09-13T13:06:35.345Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:34.968" grpc.time_ms=376.929 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1823gitlab # [ 418.517493] gitaly[919]: time="2026-09-13T13:06:35.406Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:35.405" grpc.time_ms=1.149 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1824gitlab # [ 418.581472] gitaly[919]: time="2026-09-13T13:06:35.470Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:35.468" grpc.time_ms=1.141 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1825gitlab # [ 418.625223] gitaly[919]: time="2026-09-13T13:06:35.514Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:35.512" grpc.time_ms=1.036 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1826gitlab # [ 418.744848] gitaly[919]: time="2026-09-13T13:06:35.631Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2DE0VH64JMW9MMND5PDBXRD duration_ms=65ns method=POST pid=919 status=200 url="http://unix/api/v4/internal/post_receive"1827gitlab # [ 418.752338] gitaly[919]: time="2026-09-13T13:06:35.632Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:35.564" grpc.time_ms=67.656 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1828gitlab # To http://gitlab/root/test-flake.git1829gitlab # [ 418.765580] gitaly[919]: time="2026-09-13T13:06:35.644Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD diskcache=b77e1120-e062-4b67-8e44-25f9c06124aa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:34.801" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1830gitlab # aa72696..d7f612c [ 418.778673] gitlab-workhorse[819]: time="2026-09-13T13:06:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521831gitlab # master -> master[ 418.780837] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:35 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 9821832gitlab # 1833gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.55 seconds)1834gitlab # [ 418.783287] gitaly[919]: time="2026-09-13T13:06:35.645Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=407 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13449 command.oublock=32 command.real_time_ms=835 command.spawn_token_fork_ms=1 command.system_time_ms=237 command.user_time_ms=170 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:34.801" grpc.time_ms=843.844 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1835gitlab: must succeed: git -C /tmp/test-flake rev-parse master1836gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1837gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1838gitlab # [ 418.961934] nixbot[3565]: INFO: ::ffff:127.0.0.1:34576 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1839gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.15 seconds)1840gitlab # [ 419.402189] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:36.289Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c87b61878874651216a89c1e","created_at":"2026-09-13T13:05:05.615Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:05.657Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c87b61878874651216a89c1e: start","job_status":"start","queue_duration_s":90.631815,"scheduling_latency_s":90.631867}1841gitlab # [ 419.748116] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:36.635Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c87b61878874651216a89c1e","created_at":"2026-09-13T13:05:05.615Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:05.657Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c87b61878874651216a89c1e: done: 0.346338 sec","job_status":"done","queue_duration_s":90.631815,"scheduling_latency_s":90.631867,"redis_calls":23,"redis_duration_s":0.010673,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.005566,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002979,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002128,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.204233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.2039898,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.346338,"completed_at":"2026-09-13T13:06:36.635Z","load_balancing_strategy":"primary","db_duration_s":0.105155,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1842gitlab # [ 419.844197] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:36.639Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"70fcccec5192f6f2e2653cdb","created_at":"2026-09-13T13:05:05.674Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:05.680Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-70fcccec5192f6f2e2653cdb: start","job_status":"start","queue_duration_s":90.958668,"scheduling_latency_s":90.958713}1843gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1844gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.06 seconds)1845gitlab # [ 419.937828] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:36.825Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"70fcccec5192f6f2e2653cdb","created_at":"2026-09-13T13:05:05.674Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:05.680Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-70fcccec5192f6f2e2653cdb: done: 0.186631 sec","job_status":"done","queue_duration_s":90.958668,"scheduling_latency_s":90.958713,"redis_calls":14,"redis_duration_s":0.004851,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001079,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002156,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001616,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.138367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14284346,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.186631,"completed_at":"2026-09-13T13:06:36.825Z","load_balancing_strategy":"primary","db_duration_s":0.037684,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1846gitlab # [ 420.042916] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:36.828Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"15c1726c271c1f21ca9aa388","created_at":"2026-09-13T13:05:06.481Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:06.487Z","job_size_bytes":5,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-15c1726c271c1f21ca9aa388: start","job_status":"start","queue_duration_s":90.340644,"scheduling_latency_s":90.34067}1847gitlab # [ 420.071969] nixbot[3565]: INFO: ::ffff:127.0.0.1:34586 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1848gitlab # [ 420.160743] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:37.049Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"15c1726c271c1f21ca9aa388","created_at":"2026-09-13T13:05:06.481Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:06.487Z","job_size_bytes":5,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-15c1726c271c1f21ca9aa388: done: 0.220897 sec","job_status":"done","queue_duration_s":90.340644,"scheduling_latency_s":90.34067,"redis_calls":12,"redis_duration_s":0.0034549999999999997,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000829,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002064,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000562,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047162,"duration_s":0.220897,"completed_at":"2026-09-13T13:06:37.049Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044342,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1849gitlab # [ 420.253178] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:37.051Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f40a420e6523c6e17c69190e","created_at":"2026-09-13T13:05:06.524Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:06.531Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f40a420e6523c6e17c69190e: start","job_status":"start","queue_duration_s":90.520047,"scheduling_latency_s":90.520067}1850gitlab # [ 420.430312] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:37.317Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"f40a420e6523c6e17c69190e","created_at":"2026-09-13T13:05:06.524Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:06.531Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-f40a420e6523c6e17c69190e: done: 0.266348 sec","job_status":"done","queue_duration_s":90.520047,"scheduling_latency_s":90.520067,"redis_calls":11,"redis_duration_s":0.00323,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000748,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001898,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000584,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.225,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.225,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.167804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055296,"duration_s":0.266348,"completed_at":"2026-09-13T13:06:37.317Z","load_balancing_strategy":"primary","db_duration_s":0.101974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1851gitlab # [ 420.528266] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:37.319Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"4b5b4742d368bddca88e926b","created_at":"2026-09-13T13:05:06.534Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:06.539Z","job_size_bytes":89,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-4b5b4742d368bddca88e926b: start","job_status":"start","queue_duration_s":90.779902,"scheduling_latency_s":90.779929}1852gitlab # [ 420.553943] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:37.367Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"4b5b4742d368bddca88e926b","created_at":"2026-09-13T13:05:06.534Z","trace_propagation_headers":{"sentry-trace":"081e7dc78d2742a0a89137b3d256a92f-de510f4cf6e040fc","baggage":"sentry-trace_id=081e7dc78d2742a0a89137b3d256a92f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDY37KBWN4SBRX5XT0AGBN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:06.539Z","job_size_bytes":89,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-4b5b4742d368bddca88e926b: done: 0.047423 sec","job_status":"done","queue_duration_s":90.779902,"scheduling_latency_s":90.779929,"redis_calls":11,"redis_duration_s":0.003274,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000926,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001652,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004233,"duration_s":0.047423,"completed_at":"2026-09-13T13:06:37.366Z","load_balancing_strategy":"primary","db_duration_s":0.008586,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1853gitlab # [ 420.603176] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:37.368Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 aa726960fcb15de306f9a352e35de564cfe41bbb refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"41be11c917e96bb572ce2a24","created_at":"2026-09-13T13:05:16.939Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-666b6b3911aa4dc6","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:05:16.947Z","job_size_bytes":185,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-41be11c917e96bb572ce2a24: start","job_status":"start","queue_duration_s":80.421073,"scheduling_latency_s":80.421093}1854gitlab # [ 420.619559] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:37.387Z","message":"changes: \"0000000000000000000000000000000000000000 aa726960fcb15de306f9a352e35de564cfe41bbb refs/heads/master\\n\""}1855gitlab # [ 420.622379] gitaly[919]: time="2026-09-13T13:06:37.437Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:47.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:37.420" grpc.time_ms=17.628 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1856gitlab # [ 420.635766] gitaly[919]: time="2026-09-13T13:06:37.463Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:06:47.037" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:37.447" grpc.time_ms=15.567 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1857gitlab # [ 420.686509] gitaly[919]: time="2026-09-13T13:06:37.575Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T13:06:47.147" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:37.558" grpc.time_ms=17.269 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1858gitlab # [ 420.751334] gitaly[919]: time="2026-09-13T13:06:37.640Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:37.639" grpc.time_ms=0.787 pid=919 span.kind=server system=grpc1859gitlab # [ 420.808220] gitaly[919]: time="2026-09-13T13:06:37.697Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:37.696" grpc.time_ms=0.82 pid=919 span.kind=server system=grpc1860gitlab # [ 420.830123] gitaly[919]: time="2026-09-13T13:06:37.714Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T diskcache=e81474ae-f574-4360-8f29-39a0fe7876d7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T13:06:47.186" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:37.596" pid=919 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1861gitlab # [ 420.849570] gitaly[919]: time="2026-09-13T13:06:37.715Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=122 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4975 command.oublock=8 command.real_time_ms=111 command.spawn_token_fork_ms=1 command.system_time_ms=94 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T13:06:47.186" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:37.596" grpc.time_ms=118.347 pid=919 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1862gitlab # [ 420.870039] gitaly[919]: time="2026-09-13T13:06:37.746Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T13:06:47.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:37.746" grpc.time_ms=0.34 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863gitlab # [ 420.892142] gitaly[919]: time="2026-09-13T13:06:37.781Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=858 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T13:06:47.015" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:37.756" grpc.time_ms=24.486 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1865gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.08 seconds)1866gitlab # [ 421.086929] gitaly[919]: time="2026-09-13T13:06:37.975Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-13T13:07:07.061" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:37.961" grpc.time_ms=13.644 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 421.135165] nixbot[3565]: INFO: ::ffff:127.0.0.1:54724 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1868gitlab # [ 421.147932] gitaly[919]: time="2026-09-13T13:06:38.036Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T13:06:48.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:38.036" grpc.time_ms=0.597 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1869gitlab # [ 421.175245] gitaly[919]: time="2026-09-13T13:06:38.054Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T13:06:48.011" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T13:06:38.041" grpc.time_ms=12.622 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1870gitlab # [ 421.215221] gitaly[919]: time="2026-09-13T13:06:38.104Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:48.054" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:38.084" grpc.time_ms=19.298 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1871gitlab # [ 421.248338] gitaly[919]: time="2026-09-13T13:06:38.137Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:48.086" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:38.117" grpc.time_ms=19.11 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1872gitlab # [ 421.307993] gitaly[919]: time="2026-09-13T13:06:38.196Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T13:07:08.066" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:38.166" grpc.time_ms=30.111 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 421.924318] gitaly[919]: time="2026-09-13T13:06:38.812Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T13:06:48.069" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:38.810" grpc.time_ms=2.183 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 421.973792] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:38.860Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 aa726960fcb15de306f9a352e35de564cfe41bbb refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"41be11c917e96bb572ce2a24","created_at":"2026-09-13T13:05:16.939Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-666b6b3911aa4dc6","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:05:16.947Z","job_size_bytes":185,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-41be11c917e96bb572ce2a24: done: 1.491487 sec","job_status":"done","queue_duration_s":80.421073,"scheduling_latency_s":80.421093,"gitaly_calls":13,"gitaly_duration_s":0.35444,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.069954,"redis_read_bytes":2103,"redis_write_bytes":21747,"redis_cache_calls":15,"redis_cache_duration_s":0.006901,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008675,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.003056,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7835,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.007132,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.028917,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.015273,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.091,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.327,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.378,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.091,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.983193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00082962,"exclusive_lock_hold_duration_s":0.04110584,"duration_s":1.491487,"completed_at":"2026-09-13T13:06:38.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.1642,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1875gitlab # [ 422.076358] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:38.863Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"09fe435d891ed3d38e22c2bf","created_at":"2026-09-13T13:05:22.465Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-0cb23448aac5485f","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:22.480Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-09fe435d891ed3d38e22c2bf: start","job_status":"start","queue_duration_s":76.383394,"scheduling_latency_s":76.38344}1876gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1877gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.06 seconds)1878gitlab # [ 422.143038] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:39.030Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"09fe435d891ed3d38e22c2bf","created_at":"2026-09-13T13:05:22.465Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-0cb23448aac5485f","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:22.480Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-09fe435d891ed3d38e22c2bf: done: 0.167038 sec","job_status":"done","queue_duration_s":76.383394,"scheduling_latency_s":76.38344,"redis_calls":16,"redis_duration_s":0.006644,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003424,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00244,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00078,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007481,"duration_s":0.167038,"completed_at":"2026-09-13T13:06:39.030Z","load_balancing_strategy":"primary","db_duration_s":0.041352,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1879gitlab # [ 422.242401] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:39.034Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f0c3b2f793c8c5606ad68e45","created_at":"2026-09-13T13:05:22.486Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-0cb23448aac5485f","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:22.497Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f0c3b2f793c8c5606ad68e45: start","job_status":"start","queue_duration_s":76.536328,"scheduling_latency_s":76.53636}1880gitlab # [ 422.291403] nixbot[3565]: INFO: ::ffff:127.0.0.1:54728 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1881gitlab # [ 422.331057] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:39.218Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"f0c3b2f793c8c5606ad68e45","created_at":"2026-09-13T13:05:22.486Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-0cb23448aac5485f","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:22.497Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-f0c3b2f793c8c5606ad68e45: done: 0.184766 sec","job_status":"done","queue_duration_s":76.536328,"scheduling_latency_s":76.53636,"redis_calls":21,"redis_duration_s":0.008156,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003321,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002281,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.116191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00106838,"duration_s":0.184766,"completed_at":"2026-09-13T13:06:39.218Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1882gitlab # [ 422.428899] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:39.223Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0cca39d17ca619e853983a9f","created_at":"2026-09-13T13:05:22.713Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-0cb23448aac5485f","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:22.722Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0cca39d17ca619e853983a9f: start","job_status":"start","queue_duration_s":76.500269,"scheduling_latency_s":76.500315}1883gitlab # [ 422.816806] gitaly[919]: time="2026-09-13T13:06:39.705Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:39.703" grpc.time_ms=1.972 pid=919 span.kind=server system=grpc1884gitlab # [ 422.873263] gitaly[919]: time="2026-09-13T13:06:39.762Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:39.761" grpc.time_ms=0.724 pid=919 span.kind=server system=grpc1885gitlab # [ 422.927418] gitaly[919]: time="2026-09-13T13:06:39.815Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:39.814" grpc.time_ms=0.77 pid=919 span.kind=server system=grpc1886gitlab # [ 422.952199] gitaly[919]: time="2026-09-13T13:06:39.828Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 diskcache=9d23966c-bad0-4d5c-b478-ba2a3f8a9b55 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:09.545" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:39.646" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1887gitlab # [ 422.971590] gitaly[919]: time="2026-09-13T13:06:39.828Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=205 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7528 command.oublock=0 command.real_time_ms=175 command.spawn_token_fork_ms=1 command.system_time_ms=93 command.user_time_ms=112 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:09.545" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:39.646" grpc.time_ms=182.013 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1888gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1889gitlab # [ 423.163366] gitaly[919]: time="2026-09-13T13:06:40.052Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T13:07:10.049" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.050" grpc.time_ms=1.687 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.06 seconds)1891gitlab # [ 423.220480] nixbot[3565]: INFO: ::ffff:127.0.0.1:54732 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1892gitlab # [ 423.247327] gitaly[919]: time="2026-09-13T13:06:40.136Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYMEW3157B8F3WD8C5F84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:10.133" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.134" grpc.time_ms=1.476 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893gitlab # [ 423.292090] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.179Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"0cca39d17ca619e853983a9f","created_at":"2026-09-13T13:05:22.713Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-0cb23448aac5485f","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T13:05:22.722Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0cca39d17ca619e853983a9f: done: 0.956203 sec","job_status":"done","queue_duration_s":76.500269,"scheduling_latency_s":76.500315,"gitaly_calls":3,"gitaly_duration_s":0.196211,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.043242,"redis_read_bytes":455,"redis_write_bytes":27998,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00214,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002487,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.015789,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.006357,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14492,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.008948,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000844,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00094,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003597,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.109,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.196,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.202,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.057,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.616692,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.8311649,"duration_s":0.956203,"completed_at":"2026-09-13T13:06:40.179Z","load_balancing_strategy":"primary","db_duration_s":0.126739,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1894gitlab # [ 423.377592] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.182Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","aa726960fcb15de306f9a352e35de564cfe41bbb d7f612ca5106e30c4e4733fb3b372a4004ee28e2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3b4e64070530b6b367d34959","created_at":"2026-09-13T13:06:35.606Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-8387a8a84a9344f2","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:35.611Z","job_size_bytes":185,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3b4e64070530b6b367d34959: start","job_status":"start","queue_duration_s":4.571068,"scheduling_latency_s":4.571107}1895gitlab # [ 423.395394] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.205Z","message":"changes: \"aa726960fcb15de306f9a352e35de564cfe41bbb d7f612ca5106e30c4e4733fb3b372a4004ee28e2 refs/heads/master\\n\""}1896gitlab # [ 423.397899] gitaly[919]: time="2026-09-13T13:06:40.257Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:50.210" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:40.240" grpc.time_ms=17.154 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab # [ 423.411049] gitaly[919]: time="2026-09-13T13:06:40.281Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T13:06:50.235" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:40.266" grpc.time_ms=15.393 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 423.425161] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.289Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"a14838e779a86cd0f7aac8bd","created_at":"2026-09-13T13:06:40.286Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-58d14142cca94450","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"b6357bb8e953ad81343dcff7","job_size_bytes":46,"pid":3435,"job_status":"deduplicated","message":"ProjectCacheWorker JID-a14838e779a86cd0f7aac8bd: deduplicated: until executing","deduplication.type":"until executing"}1899gitlab # [ 423.440939] gitaly[919]: time="2026-09-13T13:06:40.329Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T13:06:50.282" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:40.312" grpc.time_ms=17.303 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 423.467657] gitaly[919]: time="2026-09-13T13:06:40.356Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-13T13:07:10.338" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.338" grpc.time_ms=17.794 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab # [ 423.496962] gitaly[919]: time="2026-09-13T13:06:40.385Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=321 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T13:06:50.004" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T13:06:40.375" grpc.time_ms=10.312 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1902gitlab # [ 423.535436] gitaly[919]: time="2026-09-13T13:06:40.419Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:50.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:40.404" grpc.time_ms=15.31 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1903gitlab # [ 423.701396] gitaly[919]: time="2026-09-13T13:06:40.590Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T13:06:50.217" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.587" grpc.time_ms=2.219 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 423.740682] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.627Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","aa726960fcb15de306f9a352e35de564cfe41bbb d7f612ca5106e30c4e4733fb3b372a4004ee28e2 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3b4e64070530b6b367d34959","created_at":"2026-09-13T13:06:35.606Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-8387a8a84a9344f2","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:35.611Z","job_size_bytes":185,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3b4e64070530b6b367d34959: done: 0.445121 sec","job_status":"done","queue_duration_s":4.571068,"scheduling_latency_s":4.571107,"gitaly_calls":7,"gitaly_duration_s":0.122222,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.035716,"redis_read_bytes":501,"redis_write_bytes":11940,"redis_cache_calls":9,"redis_cache_duration_s":0.004246,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001788,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5717,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006176,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.012414,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.011092,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.04,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.037,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.037,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.278501,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00149326,"exclusive_lock_hold_duration_s":0.03458374,"duration_s":0.445121,"completed_at":"2026-09-13T13:06:40.627Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041427,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1905gitlab # [ 423.826865] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.630Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b6357bb8e953ad81343dcff7","created_at":"2026-09-13T13:06:37.468Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:37.504Z","job_size_bytes":46,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b6357bb8e953ad81343dcff7: start","job_status":"start","queue_duration_s":3.125428,"scheduling_latency_s":3.125464}1906gitlab # [ 423.844920] gitaly[919]: time="2026-09-13T13:06:40.711Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-13T19:06:40.700" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:40.701" grpc.time_ms=9.281 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 423.856637] gitaly[919]: time="2026-09-13T13:06:40.745Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T13:07:10.727" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.728" grpc.time_ms=16.782 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 423.901761] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.790Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"b6357bb8e953ad81343dcff7","created_at":"2026-09-13T13:06:37.468Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:37.504Z","job_size_bytes":46,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-b6357bb8e953ad81343dcff7: done: 0.159897 sec","job_status":"done","queue_duration_s":3.125428,"scheduling_latency_s":3.125464,"gitaly_calls":2,"gitaly_duration_s":0.033438,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010473999999999999,"redis_read_bytes":69,"redis_write_bytes":4882,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002062,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000899,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002186,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004238,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001089,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00046488,"duration_s":0.159897,"completed_at":"2026-09-13T13:06:40.789Z","load_balancing_strategy":"primary","db_duration_s":0.017644,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1909gitlab # [ 423.969052] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.792Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8200e84d9f35526e279fae04","created_at":"2026-09-13T13:06:37.536Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:37.549Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8200e84d9f35526e279fae04: start","job_status":"start","queue_duration_s":3.242599,"scheduling_latency_s":3.242628}1910gitlab # [ 423.991060] gitaly[919]: time="2026-09-13T13:06:40.856Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T diskcache=f2319a57-a216-4555-affc-370eeb4da093 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T19:06:40.824" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.825" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab # [ 424.002459] gitaly[919]: time="2026-09-13T13:06:40.857Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=3 command.system_time_ms=12 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T19:06:40.824" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.825" grpc.time_ms=31.679 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 424.052884] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.940Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"8200e84d9f35526e279fae04","created_at":"2026-09-13T13:06:37.536Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:37.549Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-8200e84d9f35526e279fae04: done: 0.148446 sec","job_status":"done","queue_duration_s":3.242599,"scheduling_latency_s":3.242628,"gitaly_calls":1,"gitaly_duration_s":0.034956,"redis_calls":15,"redis_duration_s":0.005382,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00083,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.043,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078251,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1074153,"duration_s":0.148446,"completed_at":"2026-09-13T13:06:40.940Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027523,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1913gitlab # [ 424.116406] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:40.943Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0f2db0543df887cfed13e031","created_at":"2026-09-13T13:06:37.736Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:37.740Z","job_size_bytes":23,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0f2db0543df887cfed13e031: start","job_status":"start","queue_duration_s":3.203123,"scheduling_latency_s":3.203159}1914gitlab # [ 424.131971] gitaly[919]: time="2026-09-13T13:06:41.011Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T13:07:10.893" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:40.994" grpc.time_ms=16.81 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab # [ 424.145238] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:41.031Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0f2db0543df887cfed13e031","created_at":"2026-09-13T13:06:37.736Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:37.740Z","job_size_bytes":23,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0f2db0543df887cfed13e031: done: 0.087949 sec","job_status":"done","queue_duration_s":3.203123,"scheduling_latency_s":3.203159,"gitaly_calls":1,"gitaly_duration_s":0.020024,"redis_calls":16,"redis_duration_s":0.00652,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000732,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001974,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002537,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001277,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087949,"completed_at":"2026-09-13T13:06:41.031Z","load_balancing_strategy":"primary","db_duration_s":0.005096,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1916gitlab # [ 424.189985] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:41.036Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"79fb527138e54bd86798ae9c","created_at":"2026-09-13T13:06:38.606Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1d838557fc4e3b20bac2e97b7ca24a26c89010801dd92396e411a241320798f3","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:38.613Z","job_size_bytes":467,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-79fb527138e54bd86798ae9c: start","job_status":"start","queue_duration_s":2.423479,"scheduling_latency_s":2.423525}1917gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1918gitlab # [ 424.213087] gitaly[919]: time="2026-09-13T13:06:41.102Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:11.100" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:41.100" grpc.time_ms=1.009 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 424.251013] gitaly[919]: time="2026-09-13T13:06:41.122Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T13:06:51.076" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:41.106" grpc.time_ms=15.508 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 424.293916] nixbot[3565]: INFO: ::ffff:127.0.0.1:54734 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1921gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.07 seconds)1922gitlab # [ 424.437808] gitaly[919]: time="2026-09-13T13:06:41.326Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T13:07:11.324" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:41.324" grpc.time_ms=1.605 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1923gitlab # [ 424.475211] gitaly[919]: time="2026-09-13T13:06:41.339Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=181 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T13:07:11.334" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:41.334" grpc.time_ms=4.597 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1924gitlab # [ 424.634759] gitaly[919]: time="2026-09-13T13:06:41.523Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:51.136" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:41.506" grpc.time_ms=16.981 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1926gitlab # [ 425.350996] nixbot[3565]: INFO: ::ffff:127.0.0.1:54742 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1927gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.06 seconds)1928gitlab # [ 425.488176] gitaly[919]: time="2026-09-13T13:06:42.376Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=11 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T13:07:12.351" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:42.351" grpc.time_ms=24.319 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 425.520502] gitaly[919]: time="2026-09-13T13:06:42.409Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=13 component=gitaly.StreamServerInterceptor correlation_id=01M2DDYEMSNDKGFTBFQ3HWDD5T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T13:07:12.383" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:42.383" grpc.time_ms=25.7 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 425.587643] sidekiq-cluster[3435]: {"severity":"WARN","time":"2026-09-13T13:06:42.475Z","class":"CreatePipelineWorker","correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"aa726960fcb15de306f9a352e35de564cfe41bbb","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"aa726960fcb15de306f9a352e35de564cfe41bbb","ref":"refs/heads/master"},"retry":0}1931gitlab # [ 425.609031] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:42.492Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"79fb527138e54bd86798ae9c","created_at":"2026-09-13T13:06:38.606Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1d838557fc4e3b20bac2e97b7ca24a26c89010801dd92396e411a241320798f3","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:38.613Z","job_size_bytes":467,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-79fb527138e54bd86798ae9c: done: 1.455458 sec","job_status":"done","queue_duration_s":2.423479,"scheduling_latency_s":2.423525,"gitaly_calls":7,"gitaly_duration_s":0.119443,"redis_calls":51,"redis_duration_s":0.017424000000000002,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.001461,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.007554,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003916,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001817,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.249315,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006386,"duration_s":1.455458,"completed_at":"2026-09-13T13:06:42.492Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.091129,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1932gitlab # [ 425.692385] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:42.495Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"ed2f3731-5cbd-4025-ace9-7efdb0a0ee00\"}"],"class":"WebHookWorker","jid":"be825e826c85717394fc8235","created_at":"2026-09-13T13:06:38.768Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99d689203600de8785ddef71338e528dc5cfa0d793bfe8d9e699011ae8168d39","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:38.775Z","job_size_bytes":1751,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-be825e826c85717394fc8235: start","job_status":"start","queue_duration_s":3.720335,"scheduling_latency_s":3.720376}1933gitlab # [ 425.835419] nixbot[3565]: INFO: ::ffff:127.0.0.1:54746 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1934gitlab # [ 425.879570] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:42.767Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"ed2f3731-5cbd-4025-ace9-7efdb0a0ee00\"}"],"class":"WebHookWorker","jid":"be825e826c85717394fc8235","created_at":"2026-09-13T13:06:38.768Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-be05ed5259b0414e","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99d689203600de8785ddef71338e528dc5cfa0d793bfe8d9e699011ae8168d39","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:38.775Z","job_size_bytes":1751,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-be825e826c85717394fc8235: done: 0.271402 sec","job_status":"done","queue_duration_s":3.720335,"scheduling_latency_s":3.720376,"redis_calls":19,"redis_duration_s":0.008345,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001701,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.001225,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003324,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002095,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09967311999997719,"cpu_s":0.145635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00078978,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.271402,"completed_at":"2026-09-13T13:06:42.767Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005657,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1935gitlab # [ 425.981925] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:42.770Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"cb1c87d186e11cca79b16011","created_at":"2026-09-13T13:06:39.621Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.629Z","job_size_bytes":23,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-cb1c87d186e11cca79b16011: start","job_status":"start","queue_duration_s":3.141122,"scheduling_latency_s":3.141167}1936gitlab # [ 426.000700] gitaly[919]: time="2026-09-13T13:06:42.880Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=9 component=gitaly.StreamServerInterceptor correlation_id=01M2DE13EP6HHK6WJBD5WCJ1EH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:42.858" grpc.time_ms=21.786 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1937gitlab # [ 426.014579] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:42 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1061938gitlab # [ 426.072790] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:42.960Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"cb1c87d186e11cca79b16011","created_at":"2026-09-13T13:06:39.621Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.629Z","job_size_bytes":23,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-cb1c87d186e11cca79b16011: done: 0.189987 sec","job_status":"done","queue_duration_s":3.141122,"scheduling_latency_s":3.141167,"redis_calls":10,"redis_duration_s":0.0046949999999999995,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000794,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002836,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001065,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.134711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00099406,"duration_s":0.189987,"completed_at":"2026-09-13T13:06:42.960Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043254,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1939gitlab # [ 426.170309] gitaly[919]: time="2026-09-13T13:06:42.998Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13J3JD3ZNQPXPP72E2MG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:42.984" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1940gitlab # [ 426.182893] gitlab-workhorse[819]: time="2026-09-13T13:06:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671941gitlab # [ 426.185092] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1181942gitlab # [ 426.187252] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:42.964Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e4d0deb5e1f677c66eb2fb79","created_at":"2026-09-13T13:06:39.838Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.850Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e4d0deb5e1f677c66eb2fb79: start","job_status":"start","queue_duration_s":3.114024,"scheduling_latency_s":3.114069}1943gitlab # [ 426.205010] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.051Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"e4d0deb5e1f677c66eb2fb79","created_at":"2026-09-13T13:06:39.838Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.850Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-e4d0deb5e1f677c66eb2fb79: done: 0.087301 sec","job_status":"done","queue_duration_s":3.114024,"scheduling_latency_s":3.114069,"redis_calls":10,"redis_duration_s":0.007751,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000864,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062824,"duration_s":0.087301,"completed_at":"2026-09-13T13:06:43.051Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1944gitlab # [ 426.248375] gitaly[919]: time="2026-09-13T13:06:42.998Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13J3JD3ZNQPXPP72E2MG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:42.984" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1945gitlab # [ 426.261817] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.055Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b4a3a77d4ae84f562cb507c1","created_at":"2026-09-13T13:06:39.876Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.891Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b4a3a77d4ae84f562cb507c1: start","job_status":"start","queue_duration_s":3.16402,"scheduling_latency_s":3.164052}1946gitlab # [ 426.277505] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.119Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"b4a3a77d4ae84f562cb507c1","created_at":"2026-09-13T13:06:39.876Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.891Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-b4a3a77d4ae84f562cb507c1: done: 0.064236 sec","job_status":"done","queue_duration_s":3.16402,"scheduling_latency_s":3.164052,"redis_calls":10,"redis_duration_s":0.004919,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000715,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003548,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000656,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004595,"duration_s":0.064236,"completed_at":"2026-09-13T13:06:43.119Z","load_balancing_strategy":"primary","db_duration_s":0.008229,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1947gitlab # [ 426.320775] gitlab-workhorse[819]: time="2026-09-13T13:06:43Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371948gitlab # [ 426.322568] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:43 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 1321949gitlab # [ 426.324459] gitaly[919]: time="2026-09-13T13:06:42.999Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13J3JD3ZNQPXPP72E2MG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:42.984" grpc.time_ms=15.035 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1950gitlab # [ 426.338466] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.121Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4a3e81d42d4a598be1d66f9f","created_at":"2026-09-13T13:06:39.922Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.934Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4a3e81d42d4a598be1d66f9f: start","job_status":"start","queue_duration_s":3.187243,"scheduling_latency_s":3.187274}1951gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1952gitlab # [ 426.353415] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"4a3e81d42d4a598be1d66f9f","created_at":"2026-09-13T13:06:39.922Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.934Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-4a3e81d42d4a598be1d66f9f: done: 0.069335 sec","job_status":"done","queue_duration_s":3.187243,"scheduling_latency_s":3.187274,"redis_calls":13,"redis_duration_s":0.004633,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000716,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00131,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000674,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049128,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00088598,"duration_s":0.069335,"completed_at":"2026-09-13T13:06:43.191Z","load_balancing_strategy":"primary","db_duration_s":0.007342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 426.396860] nixbot[3565]: {"ts": "2026-09-13T13:06:43.194Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1954gitlab # [ 426.399242] gitaly[919]: time="2026-09-13T13:06:43.136Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13NXC6M8GRF8KDKCAA8P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:43.097" pid=919 remote_ip="2001:db8:1::1" request_sha=11442b5fc7c6a6b84907bb25c3c9e94209b3e918 response_bytes=337 span.kind=server system=grpc1955gitlab # [ 426.409387] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a3bf74993a389665dc31661b","created_at":"2026-09-13T13:06:39.935Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.946Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a3bf74993a389665dc31661b: start","job_status":"start","queue_duration_s":3.246354,"scheduling_latency_s":3.246381}1956gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.07 seconds)1957gitlab # [ 426.425715] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.250Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"a3bf74993a389665dc31661b","created_at":"2026-09-13T13:06:39.935Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.946Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-a3bf74993a389665dc31661b: done: 0.057246 sec","job_status":"done","queue_duration_s":3.246354,"scheduling_latency_s":3.246381,"redis_calls":10,"redis_duration_s":0.003253,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000616,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059576,"duration_s":0.057246,"completed_at":"2026-09-13T13:06:43.250Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010933,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1958gitlab # [ 426.468809] gitaly[919]: time="2026-09-13T13:06:43.136Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13NXC6M8GRF8KDKCAA8P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:43.097" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1959gitlab # [ 426.481913] nixbot[3565]: INFO: ::ffff:127.0.0.1:54760 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK1960gitlab # [ 426.483819] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.253Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7dd14ff04e43a55b65378048","created_at":"2026-09-13T13:06:39.948Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.958Z","job_size_bytes":19,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7dd14ff04e43a55b65378048: start","job_status":"start","queue_duration_s":3.294724,"scheduling_latency_s":3.294753}1961gitlab # [ 426.500689] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.296Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7dd14ff04e43a55b65378048","created_at":"2026-09-13T13:06:39.948Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.958Z","job_size_bytes":19,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7dd14ff04e43a55b65378048: done: 0.042732 sec","job_status":"done","queue_duration_s":3.294724,"scheduling_latency_s":3.294753,"redis_calls":11,"redis_duration_s":0.004633,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000606,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002561,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001466,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026526,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047054,"duration_s":0.042732,"completed_at":"2026-09-13T13:06:43.296Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005007,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 426.543688] gitaly[919]: time="2026-09-13T13:06:43.138Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13NXC6M8GRF8KDKCAA8P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:43.097" grpc.time_ms=39.317 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1963gitlab # [ 426.558154] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.302Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"21af6224fb6fa46f946f854d","created_at":"2026-09-13T13:06:39.964Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.973Z","job_size_bytes":66,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-21af6224fb6fa46f946f854d: start","job_status":"start","queue_duration_s":3.32859,"scheduling_latency_s":3.328611}1964gitlab # [ 426.575728] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.371Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"21af6224fb6fa46f946f854d","created_at":"2026-09-13T13:06:39.964Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.973Z","job_size_bytes":66,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-21af6224fb6fa46f946f854d: done: 0.069393 sec","job_status":"done","queue_duration_s":3.32859,"scheduling_latency_s":3.328611,"redis_calls":10,"redis_duration_s":0.007871,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003292,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002878,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001701,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032362,"duration_s":0.069393,"completed_at":"2026-09-13T13:06:43.371Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006845,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 426.620599] gitaly[919]: time="2026-09-13T13:06:43.305Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:13.102" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:43.303" grpc.time_ms=1.602 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 426.633598] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.373Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c8e3286418ed477b8eacf8cd","created_at":"2026-09-13T13:06:39.976Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.983Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c8e3286418ed477b8eacf8cd: start","job_status":"start","queue_duration_s":3.389846,"scheduling_latency_s":3.389865}1967gitlab # [ 426.649676] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.449Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"c8e3286418ed477b8eacf8cd","created_at":"2026-09-13T13:06:39.976Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:39.983Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-c8e3286418ed477b8eacf8cd: done: 0.076153 sec","job_status":"done","queue_duration_s":3.389846,"scheduling_latency_s":3.389865,"redis_calls":10,"redis_duration_s":0.003137,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000783,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0008562,"duration_s":0.076153,"completed_at":"2026-09-13T13:06:43.449Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1968gitlab # [ 426.693399] gitaly[919]: time="2026-09-13T13:06:43.439Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:53.288" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:43.429" grpc.time_ms=10.052 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1969gitlab # [ 426.707700] nixbot[3565]: {"ts": "2026-09-13T13:06:43.527Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb \"HTTP/1.1 201 Created\""}1970gitlab # [ 426.710642] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3131971gitlab # [ 426.713160] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.456Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"48c70d41064760f22a840a8b","created_at":"2026-09-13T13:06:39.994Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.001Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-48c70d41064760f22a840a8b: start","job_status":"start","queue_duration_s":3.454627,"scheduling_latency_s":3.454663}1972gitlab # [ 426.729557] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.542Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"48c70d41064760f22a840a8b","created_at":"2026-09-13T13:06:39.994Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.001Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-48c70d41064760f22a840a8b: done: 0.086521 sec","job_status":"done","queue_duration_s":3.454627,"scheduling_latency_s":3.454663,"redis_calls":10,"redis_duration_s":0.007483999999999999,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001685,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00071198,"duration_s":0.086521,"completed_at":"2026-09-13T13:06:43.542Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016853,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1973gitlab # [ 426.773309] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.545Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e61e4551be41f89887552f94","created_at":"2026-09-13T13:06:40.002Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.010Z","job_size_bytes":6,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e61e4551be41f89887552f94: start","job_status":"start","queue_duration_s":3.535643,"scheduling_latency_s":3.535677}1974gitlab # [ 426.789211] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.597Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e61e4551be41f89887552f94","created_at":"2026-09-13T13:06:40.002Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.010Z","job_size_bytes":6,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e61e4551be41f89887552f94: done: 0.051641 sec","job_status":"done","queue_duration_s":3.535643,"scheduling_latency_s":3.535677,"redis_calls":10,"redis_duration_s":0.008074,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000675,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002246,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029404,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00082456,"duration_s":0.051641,"completed_at":"2026-09-13T13:06:43.597Z","load_balancing_strategy":"primary","db_duration_s":0.004895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1975gitlab # [ 426.833461] gitaly[919]: time="2026-09-13T13:06:43.647Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE146DE62Q6WEAVPGM8XW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:13.245" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:43.646" grpc.time_ms=0.867 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1976gitlab # [ 426.846310] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.601Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f553a1ed8b82d383a5bdb733","created_at":"2026-09-13T13:06:40.011Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.017Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f553a1ed8b82d383a5bdb733: start","job_status":"start","queue_duration_s":3.58416,"scheduling_latency_s":3.584192}1977gitlab # [ 426.861595] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.668Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"f553a1ed8b82d383a5bdb733","created_at":"2026-09-13T13:06:40.011Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.017Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-f553a1ed8b82d383a5bdb733: done: 0.066894 sec","job_status":"done","queue_duration_s":3.58416,"scheduling_latency_s":3.584192,"redis_calls":10,"redis_duration_s":0.004163,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001319,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001891,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006866,"duration_s":0.066894,"completed_at":"2026-09-13T13:06:43.668Z","load_balancing_strategy":"primary","db_duration_s":0.013999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 426.905097] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.671Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"8c266b36a1f83ed0e5648d51","created_at":"2026-09-13T13:06:40.026Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.034Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-8c266b36a1f83ed0e5648d51: start","job_status":"start","queue_duration_s":3.63647,"scheduling_latency_s":3.6365}1979gitlab # [ 426.922627] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.794Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"8c266b36a1f83ed0e5648d51","created_at":"2026-09-13T13:06:40.026Z","trace_propagation_headers":{"sentry-trace":"7d0eb04321ed42f8ab5b433a9b7fc9d9-4936fe89043143e6","baggage":"sentry-trace_id=7d0eb04321ed42f8ab5b433a9b7fc9d9,sentry-release=Unknown"},"correlation_id":"01M2DDYMEW3157B8F3WD8C5F84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.034Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-8c266b36a1f83ed0e5648d51: done: 0.123468 sec","job_status":"done","queue_duration_s":3.63647,"scheduling_latency_s":3.6365,"redis_calls":14,"redis_duration_s":0.005655,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000502,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002806,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00100624,"duration_s":0.123468,"completed_at":"2026-09-13T13:06:43.794Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027409,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980gitlab # [ 426.967236] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.798Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"173eb42be6d7f560112e0772","created_at":"2026-09-13T13:06:40.300Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-58d14142cca94450","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8200e84d9f35526e279fae04","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.304Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-173eb42be6d7f560112e0772: start","job_status":"start","queue_duration_s":3.493279,"scheduling_latency_s":3.493324}1981gitlab # [ 426.984247] gitaly[919]: time="2026-09-13T13:06:43.813Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE146DE62Q6WEAVPGM8XW7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:53.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:43.800" grpc.time_ms=13.144 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1982gitlab # [ 426.997567] gitaly[919]: time="2026-09-13T13:06:43.857Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD diskcache=d35495e9-8442-49ce-91aa-9706e4f431e0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T19:06:43.840" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:43.842" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 427.007517] gitaly[919]: time="2026-09-13T13:06:43.859Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=291 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T19:06:43.840" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:43.842" grpc.time_ms=17.074 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 427.021046] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:43 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3601985gitlab # [ 427.023524] nixbot[3565]: {"ts": "2026-09-13T13:06:43.896Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb \"HTTP/1.1 201 Created\""}1986gitlab # [ 427.026432] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.894Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"173eb42be6d7f560112e0772","created_at":"2026-09-13T13:06:40.300Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-58d14142cca94450","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"8200e84d9f35526e279fae04","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.304Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-173eb42be6d7f560112e0772: done: 0.096932 sec","job_status":"done","queue_duration_s":3.493279,"scheduling_latency_s":3.493324,"gitaly_calls":1,"gitaly_duration_s":0.021738,"redis_calls":12,"redis_duration_s":0.006953,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002412,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001787,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0546731,"duration_s":0.096932,"completed_at":"2026-09-13T13:06:43.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1987gitlab # [ 427.071445] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:43.899Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"71b7625c7898ea5e4abd453c","created_at":"2026-09-13T13:06:40.511Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-58d14142cca94450","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d0f9a8d09adc9182c6a90b4bffde49f0f98e19bf545ce1dbd15cba654803a56f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.519Z","job_size_bytes":467,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-71b7625c7898ea5e4abd453c: start","job_status":"start","queue_duration_s":3.379721,"scheduling_latency_s":3.379759}1988gitlab # [ 427.088788] gitaly[919]: time="2026-09-13T13:06:43.946Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:13.845" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:43.945" grpc.time_ms=0.691 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989gitlab # [ 427.101169] gitaly[919]: time="2026-09-13T13:06:43.966Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T13:06:53.009" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:43.950" grpc.time_ms=15.828 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 427.115043] gitaly[919]: time="2026-09-13T13:06:43.997Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE14J36G13XF0MWRR091SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:13.496" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:43.996" grpc.time_ms=0.668 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1991gitlab # [ 427.145438] gitaly[919]: time="2026-09-13T13:06:44.034Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T13:07:14.032" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:44.032" grpc.time_ms=1.252 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1992gitlab # [ 427.180515] gitaly[919]: time="2026-09-13T13:06:44.042Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=50 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T13:07:14.038" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:44.038" grpc.time_ms=3.372 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1993gitlab # [ 427.208367] gitaly[919]: time="2026-09-13T13:06:44.065Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2DE14J36G13XF0MWRR091SP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:54.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:44.047" grpc.time_ms=17.91 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994gitlab # [ 427.226130] gitaly[919]: time="2026-09-13T13:06:44.090Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:54.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:44.075" grpc.time_ms=15.242 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1995gitlab # [ 427.249070] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2301996gitlab # [ 427.253101] nixbot[3565]: {"ts": "2026-09-13T13:06:44.139Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/aa726960fcb15de306f9a352e35de564cfe41bbb \"HTTP/1.1 201 Created\""}1997gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'1998gitlab # [ 427.408565] gitaly[919]: time="2026-09-13T13:06:44.297Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DE14V7T0XZERF3SRG5ARKN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:44.281" grpc.time_ms=15.542 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc1999gitlab # [ 427.446784] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 982000gitlab # [ 427.503645] nixbot[3565]: INFO: ::ffff:127.0.0.1:54762 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK2001gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.09 seconds)2002gitlab # [ 427.512255] gitaly[919]: time="2026-09-13T13:06:44.392Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE14YC8AQTQQAWD9WKWEVQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:44.372" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2003gitlab # [ 427.526657] gitlab-workhorse[819]: time="2026-09-13T13:06:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672004gitlab # [ 427.529091] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 932005gitlab # [ 427.531357] gitaly[919]: time="2026-09-13T13:06:44.392Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE14YC8AQTQQAWD9WKWEVQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:44.372" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2006gitlab # [ 427.547551] gitaly[919]: time="2026-09-13T13:06:44.392Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE14YC8AQTQQAWD9WKWEVQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:44.372" grpc.time_ms=20.385 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2007gitlab # [ 427.935363] gitaly[919]: time="2026-09-13T13:06:44.824Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=879 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T13:07:14.796" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:44.797" grpc.time_ms=26.54 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2008gitlab # [ 427.980403] gitaly[919]: time="2026-09-13T13:06:44.851Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE0VH64JMW9MMND5PDBXRD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T13:07:14.829" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:44.829" grpc.time_ms=21.467 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2009gitlab # [ 428.007551] sidekiq-cluster[3435]: {"severity":"WARN","time":"2026-09-13T13:06:44.880Z","class":"CreatePipelineWorker","correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"aa726960fcb15de306f9a352e35de564cfe41bbb","after":"d7f612ca5106e30c4e4733fb3b372a4004ee28e2","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"d7f612ca5106e30c4e4733fb3b372a4004ee28e2","ref":"refs/heads/master"},"retry":0}2010gitlab # [ 428.018796] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:44.894Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"71b7625c7898ea5e4abd453c","created_at":"2026-09-13T13:06:40.511Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-58d14142cca94450","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d0f9a8d09adc9182c6a90b4bffde49f0f98e19bf545ce1dbd15cba654803a56f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.519Z","job_size_bytes":467,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-71b7625c7898ea5e4abd453c: done: 0.995515 sec","job_status":"done","queue_duration_s":3.379721,"scheduling_latency_s":3.379759,"gitaly_calls":7,"gitaly_duration_s":0.106912,"redis_calls":33,"redis_duration_s":0.009252,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.00054,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003383,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.00246,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002869,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.855459,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.995515,"completed_at":"2026-09-13T13:06:44.894Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033555,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2011gitlab # [ 428.073697] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:44.898Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"dca2c153-9612-42dd-a529-ab47bedf662c\"}"],"class":"WebHookWorker","jid":"f3fcaa11cae5c1d025ba24a6","created_at":"2026-09-13T13:06:40.553Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-58d14142cca94450","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:58f208d48f69d12900a34a3ccabcbcf5dfadeaa28d02c443a509da62ef5c6cf7","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.558Z","job_size_bytes":1749,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f3fcaa11cae5c1d025ba24a6: start","job_status":"start","queue_duration_s":4.340067,"scheduling_latency_s":4.34012}2012gitlab # [ 428.091290] nixbot[3565]: INFO: ::ffff:127.0.0.1:54776 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2013gitlab # [ 428.092604] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:44.972Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"dca2c153-9612-42dd-a529-ab47bedf662c\"}"],"class":"WebHookWorker","jid":"f3fcaa11cae5c1d025ba24a6","created_at":"2026-09-13T13:06:40.553Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-58d14142cca94450","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:58f208d48f69d12900a34a3ccabcbcf5dfadeaa28d02c443a509da62ef5c6cf7","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.558Z","job_size_bytes":1749,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f3fcaa11cae5c1d025ba24a6: done: 0.074283 sec","job_status":"done","queue_duration_s":4.340067,"scheduling_latency_s":4.34012,"redis_calls":14,"redis_duration_s":0.004586,"redis_read_bytes":25,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.000666,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002586,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001334,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.01139643999999862,"cpu_s":0.047908,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.074283,"completed_at":"2026-09-13T13:06:44.972Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2014gitlab # [ 428.141006] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:44.975Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"058e1ee4ebd433f6131cf37b","created_at":"2026-09-13T13:06:40.759Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-54ef5cbc8f4240f7","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.770Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-058e1ee4ebd433f6131cf37b: start","job_status":"start","queue_duration_s":4.205236,"scheduling_latency_s":4.205268}2015gitlab # [ 428.159411] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.047Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"058e1ee4ebd433f6131cf37b","created_at":"2026-09-13T13:06:40.759Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-54ef5cbc8f4240f7","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:40.770Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-058e1ee4ebd433f6131cf37b: done: 0.072794 sec","job_status":"done","queue_duration_s":4.205236,"scheduling_latency_s":4.205268,"redis_calls":19,"redis_duration_s":0.005125,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000972,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000594,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00255,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010479,"duration_s":0.072794,"completed_at":"2026-09-13T13:06:45.047Z","load_balancing_strategy":"primary","db_duration_s":0.011823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2016gitlab # [ 428.204004] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.049Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","be825e826c85717394fc8235"],"class":"WebHooks::LogExecutionWorker","jid":"798c4d9eac6d74da18f74af2","created_at":"2026-09-13T13:06:42.730Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-30430883ef7645b5","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0d3fbdd5a961591f2364940454ba6dcf70d9c13f6486ef3ddaf7d0cc4fe43504","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:42.743Z","job_size_bytes":2593,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-798c4d9eac6d74da18f74af2: start","job_status":"start","queue_duration_s":2.305956,"scheduling_latency_s":2.305979}2017gitlab # [ 428.220703] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.108Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","be825e826c85717394fc8235"],"class":"WebHooks::LogExecutionWorker","jid":"798c4d9eac6d74da18f74af2","created_at":"2026-09-13T13:06:42.730Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-30430883ef7645b5","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0d3fbdd5a961591f2364940454ba6dcf70d9c13f6486ef3ddaf7d0cc4fe43504","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:42.743Z","job_size_bytes":2593,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-798c4d9eac6d74da18f74af2: done: 0.05891 sec","job_status":"done","queue_duration_s":2.305956,"scheduling_latency_s":2.305979,"redis_calls":10,"redis_duration_s":0.0016220000000000002,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00034,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029194,"duration_s":0.05891,"completed_at":"2026-09-13T13:06:45.108Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2018gitlab # [ 428.264729] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1002019gitlab # [ 428.266776] gitaly[919]: time="2026-09-13T13:06:45.100Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=6 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE15M9X1M2JWRVG3F3KX38 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.085" grpc.time_ms=14.791 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2020gitlab # [ 428.279744] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.110Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"098284a49d14bb9c57d7f0f6","created_at":"2026-09-13T13:06:43.494Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-b3ed54ee531343a4","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:43.500Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-098284a49d14bb9c57d7f0f6: start","job_status":"start","queue_duration_s":1.609675,"scheduling_latency_s":1.609695}2021gitlab # [ 428.303558] gitaly[919]: time="2026-09-13T13:06:45.192Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE15QG2XFSQ2Y0VNAFQA4K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.178" pid=919 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2022gitlab # [ 428.313688] gitaly[919]: time="2026-09-13T13:06:45.192Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE15QG2XFSQ2Y0VNAFQA4K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.178" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2023gitlab # [ 428.326544] gitlab-workhorse[819]: time="2026-09-13T13:06:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672024gitlab # [ 428.328384] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:45 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 892025gitlab # [ 428.330242] gitaly[919]: time="2026-09-13T13:06:45.193Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE15QG2XFSQ2Y0VNAFQA4K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.178" grpc.time_ms=14.259 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2026gitlab # [ 428.360455] gitaly[919]: time="2026-09-13T13:06:45.249Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:45.246" grpc.time_ms=2.817 pid=919 span.kind=server system=grpc2027gitlab # [ 428.390421] gitaly[919]: time="2026-09-13T13:06:45.279Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:45.279" grpc.time_ms=0.37 pid=919 span.kind=server system=grpc2028gitlab # [ 428.427673] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 622029gitlab # [ 428.431495] gitaly[919]: time="2026-09-13T13:06:45.317Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2DE15W5W8JCEHDY33YAXNTD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.306" grpc.time_ms=9.16 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2030gitlab # [ 428.457978] gitaly[919]: time="2026-09-13T13:06:45.317Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:45.316" grpc.time_ms=0.897 pid=919 span.kind=server system=grpc2031gitlab # [ 428.474236] gitaly[919]: time="2026-09-13T13:06:45.325Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA diskcache=ae89de7e-d275-4cb7-93ee-7b132e871b45 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:15.203" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:45.203" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2032gitlab # [ 428.488410] gitaly[919]: time="2026-09-13T13:06:45.326Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=118 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7393 command.oublock=0 command.real_time_ms=117 command.spawn_token_fork_ms=0 command.system_time_ms=67 command.user_time_ms=51 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:15.203" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:45.203" grpc.time_ms=122.085 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2033gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2'2034gitlab # [ 428.506127] gitaly[919]: time="2026-09-13T13:06:45.383Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T13:07:15.381" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:45.381" grpc.time_ms=2.153 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2035gitlab # [ 428.522603] gitaly[919]: time="2026-09-13T13:06:45.402Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE15Y6604H6YHTD4E6VKRG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.380" pid=919 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2036gitlab # [ 428.534494] gitlab-workhorse[819]: time="2026-09-13T13:06:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162037gitlab # [ 428.536604] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:45 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 842038gitlab # [ 428.538686] gitaly[919]: time="2026-09-13T13:06:45.402Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE15Y6604H6YHTD4E6VKRG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.380" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2039gitlab # [ 428.553691] gitaly[919]: time="2026-09-13T13:06:45.403Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE15Y6604H6YHTD4E6VKRG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T13:06:45.380" grpc.time_ms=22.001 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc2040gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2', in 0.06 seconds)2041(finished: subtest: push delivers the webhook, in 84.55 seconds)2042subtest: eval and build post commit statuses2043gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2044gitlab # [ 428.569385] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.432Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"098284a49d14bb9c57d7f0f6","created_at":"2026-09-13T13:06:43.494Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-b3ed54ee531343a4","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:43.500Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3435,"job_status":"rescheduled","message":"PipelineProcessWorker JID-098284a49d14bb9c57d7f0f6: rescheduled"}2045gitlab # [ 428.583947] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.440Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"098284a49d14bb9c57d7f0f6","created_at":"2026-09-13T13:06:43.494Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-b3ed54ee531343a4","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:43.500Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-098284a49d14bb9c57d7f0f6: done: 0.330593 sec","job_status":"done","queue_duration_s":1.609675,"scheduling_latency_s":1.609695,"gitaly_calls":3,"gitaly_duration_s":0.134542,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014650999999999999,"redis_read_bytes":148,"redis_write_bytes":18017,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000511,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000853,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.003065,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12316,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005057,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000267,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001323,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003064,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.011,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.011,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.021,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.140598,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.28283652,"duration_s":0.330593,"completed_at":"2026-09-13T13:06:45.440Z","load_balancing_strategy":"primary","db_duration_s":0.050487,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2046gitlab # [ 428.636245] gitaly[919]: time="2026-09-13T13:06:45.422Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE13WB66EKK19XM99QBYSA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:15.418" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:45.418" grpc.time_ms=4.049 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2047gitlab # [ 428.648737] nixbot[3565]: INFO: ::ffff:127.0.0.1:54778 - "GET /api/repos/gitlab/root/test-flake/builds?commit=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 200 OK2048gitlab # [ 428.650642] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.442Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f3fcaa11cae5c1d025ba24a6"],"class":"WebHooks::LogExecutionWorker","jid":"f1b6327bf6a0a9dfb1197a38","created_at":"2026-09-13T13:06:44.957Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-bfd65249e15a49a9","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f81d0aa51e6ddf41f6545ed9e51f939ea5d52d566bfdd8557129b71983f2693e","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:44.961Z","job_size_bytes":2591,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f1b6327bf6a0a9dfb1197a38: start","job_status":"start","queue_duration_s":0.480917,"scheduling_latency_s":0.480941}2049gitlab # [ 428.667579] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.467Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f3fcaa11cae5c1d025ba24a6"],"class":"WebHooks::LogExecutionWorker","jid":"f1b6327bf6a0a9dfb1197a38","created_at":"2026-09-13T13:06:44.957Z","trace_propagation_headers":{"sentry-trace":"c93426aa61fa4995ab81ef298bf7ffc8-bfd65249e15a49a9","baggage":"sentry-trace_id=c93426aa61fa4995ab81ef298bf7ffc8,sentry-release=Unknown"},"correlation_id":"01M2DE0VH64JMW9MMND5PDBXRD","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f81d0aa51e6ddf41f6545ed9e51f939ea5d52d566bfdd8557129b71983f2693e","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:44.961Z","job_size_bytes":2591,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-f1b6327bf6a0a9dfb1197a38: done: 0.025636 sec","job_status":"done","queue_duration_s":0.480917,"scheduling_latency_s":0.480941,"redis_calls":7,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00074,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025636,"completed_at":"2026-09-13T13:06:45.467Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002687,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2050gitlab # [ 428.709954] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.469Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"83cbcec2ec04992c2b3dc96e","created_at":"2026-09-13T13:06:45.029Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-a40c590df3d34a26","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.036Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-83cbcec2ec04992c2b3dc96e: start","job_status":"start","queue_duration_s":0.432402,"scheduling_latency_s":0.432431}2051gitlab # [ 428.726540] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"83cbcec2ec04992c2b3dc96e","created_at":"2026-09-13T13:06:45.029Z","trace_propagation_headers":{"sentry-trace":"bbc5a8251e5948d4ad5591a7d5a72733-a40c590df3d34a26","baggage":"sentry-trace_id=bbc5a8251e5948d4ad5591a7d5a72733,sentry-release=Unknown"},"correlation_id":"01M2DDYEMSNDKGFTBFQ3HWDD5T","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.036Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-83cbcec2ec04992c2b3dc96e: done: 0.087847 sec","job_status":"done","queue_duration_s":0.432402,"scheduling_latency_s":0.432431,"redis_calls":11,"redis_duration_s":0.002602,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000314,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001601,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000687,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.067191,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002969,"duration_s":0.087847,"completed_at":"2026-09-13T13:06:45.557Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015256,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2052gitlab # [ 428.771056] gitaly[919]: time="2026-09-13T13:06:45.554Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:15.252" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:45.553" grpc.time_ms=0.875 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2053gitlab # [ 428.783508] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.560Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5cc0b1a4b12a45adaa1ad74d","created_at":"2026-09-13T13:06:45.195Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.197Z","job_size_bytes":23,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5cc0b1a4b12a45adaa1ad74d: start","job_status":"start","queue_duration_s":0.363133,"scheduling_latency_s":0.36316}2054gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.27 seconds)2055gitlab # [ 428.799743] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.607Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5cc0b1a4b12a45adaa1ad74d","created_at":"2026-09-13T13:06:45.195Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.197Z","job_size_bytes":23,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5cc0b1a4b12a45adaa1ad74d: done: 0.04629 sec","job_status":"done","queue_duration_s":0.363133,"scheduling_latency_s":0.36316,"redis_calls":7,"redis_duration_s":0.001712,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04629,"completed_at":"2026-09-13T13:06:45.606Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010008,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2056gitlab # [ 428.841666] gitaly[919]: time="2026-09-13T13:06:45.597Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=8 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T13:07:15.278" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:45.578" grpc.time_ms=18.313 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057gitlab # [ 428.854620] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 2192058gitlab # [ 428.857277] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.609Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6eb3413ab520924ffacb3d6b","created_at":"2026-09-13T13:06:45.331Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.333Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6eb3413ab520924ffacb3d6b: start","job_status":"start","queue_duration_s":0.275297,"scheduling_latency_s":0.275322}2059gitlab # [ 428.873494] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.640Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"6eb3413ab520924ffacb3d6b","created_at":"2026-09-13T13:06:45.331Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.333Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-6eb3413ab520924ffacb3d6b: done: 0.030932 sec","job_status":"done","queue_duration_s":0.275297,"scheduling_latency_s":0.275322,"redis_calls":7,"redis_duration_s":0.0021780000000000002,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001801,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000377,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030932,"completed_at":"2026-09-13T13:06:45.639Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003665,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2060gitlab # [ 428.915612] gitaly[919]: time="2026-09-13T13:06:45.635Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=7 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:06:55.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:45.613" grpc.time_ms=21.71 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2061gitlab # [ 428.928943] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.641Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"becec5e38829894f9862a338","created_at":"2026-09-13T13:06:45.341Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.344Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-becec5e38829894f9862a338: start","job_status":"start","queue_duration_s":0.297016,"scheduling_latency_s":0.297042}2062gitlab # [ 428.944203] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.671Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"becec5e38829894f9862a338","created_at":"2026-09-13T13:06:45.341Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.344Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-becec5e38829894f9862a338: done: 0.030233 sec","job_status":"done","queue_duration_s":0.297016,"scheduling_latency_s":0.297042,"redis_calls":7,"redis_duration_s":0.002232,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030233,"completed_at":"2026-09-13T13:06:45.671Z","load_balancing_strategy":"primary","db_duration_s":0.004077,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 428.984807] gitaly[919]: time="2026-09-13T13:06:45.654Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1642HHGF42ZNBEC0MEFF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:15.652" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:45.653" grpc.time_ms=1.07 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2064gitlab # [ 428.998552] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.673Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b3daa837bed39ad39fdc4ae7","created_at":"2026-09-13T13:06:45.353Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.356Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b3daa837bed39ad39fdc4ae7: start","job_status":"start","queue_duration_s":0.317321,"scheduling_latency_s":0.317339}2065gitlab # [ 429.014095] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.705Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b3daa837bed39ad39fdc4ae7","created_at":"2026-09-13T13:06:45.353Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.356Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b3daa837bed39ad39fdc4ae7: done: 0.032343 sec","job_status":"done","queue_duration_s":0.317321,"scheduling_latency_s":0.317339,"redis_calls":10,"redis_duration_s":0.003661,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000382,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001813,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032343,"completed_at":"2026-09-13T13:06:45.705Z","load_balancing_strategy":"primary","db_duration_s":0.003987,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2066gitlab # [ 429.056637] gitaly[919]: time="2026-09-13T13:06:45.677Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T13:06:55.065" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:45.676" grpc.time_ms=0.591 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2067gitlab # [ 429.067122] nixbot[3565]: {"ts": "2026-09-13T13:06:45.878Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 \"HTTP/1.1 201 Created\""}2068gitlab # [ 429.069833] nixbot[3565]: {"ts": "2026-09-13T13:06:45.888Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-d7f612ca5106-5"}2069gitlab # [ 429.072036] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4222070gitlab # [ 429.074466] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.706Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d201339fa0052282ba326e81","created_at":"2026-09-13T13:06:45.356Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.358Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d201339fa0052282ba326e81: start","job_status":"start","queue_duration_s":0.348438,"scheduling_latency_s":0.348456}2071gitlab # [ 429.090855] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.735Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"d201339fa0052282ba326e81","created_at":"2026-09-13T13:06:45.356Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.358Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-d201339fa0052282ba326e81: done: 0.028987 sec","job_status":"done","queue_duration_s":0.348438,"scheduling_latency_s":0.348456,"redis_calls":7,"redis_duration_s":0.00117,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028987,"completed_at":"2026-09-13T13:06:45.735Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003712,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2072gitlab # [ 429.132443] gitaly[919]: time="2026-09-13T13:06:45.707Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=3 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T13:06:55.075" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:45.686" grpc.time_ms=21.106 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2073gitlab # [ 429.145482] gitaly[919]: time="2026-09-13T13:06:45.753Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:45.752" grpc.time_ms=0.734 pid=919 span.kind=server system=grpc2074gitlab # [ 429.154992] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.737Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"116dd4e5cc5743df1045176a","created_at":"2026-09-13T13:06:45.359Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.361Z","job_size_bytes":19,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-116dd4e5cc5743df1045176a: start","job_status":"start","queue_duration_s":0.37524,"scheduling_latency_s":0.375258}2075gitlab # [ 429.170664] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.760Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"116dd4e5cc5743df1045176a","created_at":"2026-09-13T13:06:45.359Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.361Z","job_size_bytes":19,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-116dd4e5cc5743df1045176a: done: 0.023912 sec","job_status":"done","queue_duration_s":0.37524,"scheduling_latency_s":0.375258,"redis_calls":8,"redis_duration_s":0.0011480000000000001,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023912,"completed_at":"2026-09-13T13:06:45.760Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 429.213390] gitaly[919]: time="2026-09-13T13:06:45.778Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:45.777" grpc.time_ms=0.851 pid=919 span.kind=server system=grpc2077gitlab # [ 429.222756] gitaly[919]: time="2026-09-13T13:06:45.784Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 diskcache=a5d46b95-9f63-406f-b246-20ed0916b776 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T13:06:55.099" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:45.710" pid=919 ref=refs/keep-around/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 remote_ip="2001:db8:1::1" revision=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 span.kind=server system=grpc username=root2078gitlab # [ 429.233881] nixbot[3565]: {"ts": "2026-09-13T13:06:46.099Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-d7f612ca5106-5", "argv0": "bwrap"}2079gitlab # [ 429.236182] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.762Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f90dff6b5b2174a7f1febace","created_at":"2026-09-13T13:06:45.362Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.364Z","job_size_bytes":66,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f90dff6b5b2174a7f1febace: start","job_status":"start","queue_duration_s":0.397494,"scheduling_latency_s":0.397511}2080gitlab # [ 429.252715] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.783Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"f90dff6b5b2174a7f1febace","created_at":"2026-09-13T13:06:45.362Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.364Z","job_size_bytes":66,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-f90dff6b5b2174a7f1febace: done: 0.021793 sec","job_status":"done","queue_duration_s":0.397494,"scheduling_latency_s":0.397511,"redis_calls":7,"redis_duration_s":0.0009789999999999998,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000271,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021793,"completed_at":"2026-09-13T13:06:45.783Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002163,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2081gitlab # [ 429.294867] gitaly[919]: time="2026-09-13T13:06:45.785Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=65 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5085 command.oublock=8 command.real_time_ms=68 command.spawn_token_fork_ms=6 command.system_time_ms=37 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T13:06:55.099" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T13:06:45.710" grpc.time_ms=74.308 pid=919 ref=refs/keep-around/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 remote_ip="2001:db8:1::1" revision=d7f612ca5106e30c4e4733fb3b372a4004ee28e2 span.kind=server system=grpc username=root2082gitlab # [ 429.309077] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.785Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"289faa6ee8d9b38cd60849fc","created_at":"2026-09-13T13:06:45.365Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.366Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-289faa6ee8d9b38cd60849fc: start","job_status":"start","queue_duration_s":0.418274,"scheduling_latency_s":0.418291}2083gitlab # [ 429.324957] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.810Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"289faa6ee8d9b38cd60849fc","created_at":"2026-09-13T13:06:45.365Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.366Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-289faa6ee8d9b38cd60849fc: done: 0.025736 sec","job_status":"done","queue_duration_s":0.418274,"scheduling_latency_s":0.418291,"redis_calls":7,"redis_duration_s":0.001055,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000278,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025736,"completed_at":"2026-09-13T13:06:45.810Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004099,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2084gitlab # [ 429.367151] gitaly[919]: time="2026-09-13T13:06:45.788Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:15.787" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:45.787" grpc.time_ms=0.432 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2085gitlab # [ 429.379853] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.814Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e6c9710ed684e1179bbbcdfc","created_at":"2026-09-13T13:06:45.367Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.368Z","job_size_bytes":6,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e6c9710ed684e1179bbbcdfc: start","job_status":"start","queue_duration_s":0.445986,"scheduling_latency_s":0.446008}2086gitlab # [ 429.395749] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.832Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"e6c9710ed684e1179bbbcdfc","created_at":"2026-09-13T13:06:45.367Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.368Z","job_size_bytes":6,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-e6c9710ed684e1179bbbcdfc: done: 0.017874 sec","job_status":"done","queue_duration_s":0.445986,"scheduling_latency_s":0.446008,"redis_calls":7,"redis_duration_s":0.00229,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001885,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000405,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017874,"completed_at":"2026-09-13T13:06:45.832Z","load_balancing_strategy":"primary","db_duration_s":0.000408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2087gitlab # [ 429.438156] gitaly[919]: time="2026-09-13T13:06:46.073Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T13:07:16.071" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:46.072" grpc.time_ms=1.221 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2088gitlab # [ 429.450682] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.836Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"3b4b0ab50237e1076c66055d","created_at":"2026-09-13T13:06:45.369Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.371Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3b4b0ab50237e1076c66055d: start","job_status":"start","queue_duration_s":0.465287,"scheduling_latency_s":0.465312}2089gitlab # [ 429.465794] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.865Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"3b4b0ab50237e1076c66055d","created_at":"2026-09-13T13:06:45.369Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.371Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-3b4b0ab50237e1076c66055d: done: 0.029539 sec","job_status":"done","queue_duration_s":0.465287,"scheduling_latency_s":0.465312,"redis_calls":7,"redis_duration_s":0.001827,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001354,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029539,"completed_at":"2026-09-13T13:06:45.865Z","load_balancing_strategy":"primary","db_duration_s":0.004827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2090gitlab # [ 429.506782] gitaly[919]: time="2026-09-13T13:06:46.130Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE162EHXVZ1789DE5ASJJ1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:16.128" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:46.129" grpc.time_ms=0.994 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091gitlab # [ 429.518976] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.867Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"723915b40537de2cf8ee073c","created_at":"2026-09-13T13:06:45.433Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.435Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-723915b40537de2cf8ee073c: start","job_status":"start","queue_duration_s":0.432416,"scheduling_latency_s":0.432434}2092gitlab # [ 429.534104] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.898Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"723915b40537de2cf8ee073c","created_at":"2026-09-13T13:06:45.433Z","trace_propagation_headers":{"sentry-trace":"e45558d958f14afcb2d92a678fee975d-241a375677fb445a","baggage":"sentry-trace_id=e45558d958f14afcb2d92a678fee975d,sentry-release=Unknown"},"correlation_id":"01M2DE13WB66EKK19XM99QBYSA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.435Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-723915b40537de2cf8ee073c: done: 0.030985 sec","job_status":"done","queue_duration_s":0.432416,"scheduling_latency_s":0.432434,"redis_calls":8,"redis_duration_s":0.006142,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005818,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000324,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030985,"completed_at":"2026-09-13T13:06:45.898Z","load_balancing_strategy":"primary","db_duration_s":0.003677,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2093gitlab # [ 429.574716] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.899Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"fac784b96b2a5f03db5e61ae","created_at":"2026-09-13T13:06:45.789Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-81aac0a637fc4094","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.794Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-fac784b96b2a5f03db5e61ae: start","job_status":"start","queue_duration_s":0.105456,"scheduling_latency_s":0.105475}2094gitlab # [ 429.590985] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.935Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"fac784b96b2a5f03db5e61ae","created_at":"2026-09-13T13:06:45.789Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-81aac0a637fc4094","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.794Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-fac784b96b2a5f03db5e61ae: done: 0.035979 sec","job_status":"done","queue_duration_s":0.105456,"scheduling_latency_s":0.105475,"redis_calls":7,"redis_duration_s":0.005284,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001248,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004036,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022734,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035979,"completed_at":"2026-09-13T13:06:45.935Z","load_balancing_strategy":"primary","db_duration_s":0.005151,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2095gitlab # [ 429.633806] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.937Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"5795c405ceaa4c20f1b5c069","created_at":"2026-09-13T13:06:45.795Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-81aac0a637fc4094","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.798Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-5795c405ceaa4c20f1b5c069: start","job_status":"start","queue_duration_s":0.138581,"scheduling_latency_s":0.1386}2096gitlab # [ 429.650808] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.989Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"5795c405ceaa4c20f1b5c069","created_at":"2026-09-13T13:06:45.795Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-81aac0a637fc4094","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.798Z","job_size_bytes":65,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-5795c405ceaa4c20f1b5c069: done: 0.052458 sec","job_status":"done","queue_duration_s":0.138581,"scheduling_latency_s":0.1386,"redis_calls":10,"redis_duration_s":0.002368,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001593,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000775,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052458,"completed_at":"2026-09-13T13:06:45.989Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2097gitlab # [ 429.693595] nix-daemon[3738]: accepted connection from pid 4745, user nixbot2098gitlab # [ 429.694608] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:45.990Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4da3774ac2e96632c3c9501e","created_at":"2026-09-13T13:06:45.858Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-81aac0a637fc4094","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.862Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4da3774ac2e96632c3c9501e: start","job_status":"start","queue_duration_s":0.128282,"scheduling_latency_s":0.1283}2099gitlab # [ 429.710561] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:46.155Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4da3774ac2e96632c3c9501e","created_at":"2026-09-13T13:06:45.858Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-81aac0a637fc4094","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:45.862Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-4da3774ac2e96632c3c9501e: done: 0.165114 sec","job_status":"done","queue_duration_s":0.128282,"scheduling_latency_s":0.1283,"gitaly_calls":2,"gitaly_duration_s":0.007843,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.009202,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001147,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001564,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000385,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002504,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000271,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000776,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.108991,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12283094,"duration_s":0.165114,"completed_at":"2026-09-13T13:06:46.155Z","load_balancing_strategy":"primary","db_duration_s":0.04726,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2100gitlab # [ 429.761240] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:46.159Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"318c0ae62f867669325250f5","created_at":"2026-09-13T13:06:46.052Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-9ee2c090896442c8","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:46.054Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-318c0ae62f867669325250f5: start","job_status":"start","queue_duration_s":0.104453,"scheduling_latency_s":0.10449}2101gitlab # [ 429.777116] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:46.397Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"318c0ae62f867669325250f5","created_at":"2026-09-13T13:06:46.052Z","trace_propagation_headers":{"sentry-trace":"28f1bbeaea824698a13a868bff2f693c-9ee2c090896442c8","baggage":"sentry-trace_id=28f1bbeaea824698a13a868bff2f693c,sentry-release=Unknown"},"correlation_id":"01M2DE162EHXVZ1789DE5ASJJ1","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:46.054Z","job_size_bytes":3,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-318c0ae62f867669325250f5: done: 0.238758 sec","job_status":"done","queue_duration_s":0.104453,"scheduling_latency_s":0.10449,"redis_calls":7,"redis_duration_s":0.01684,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00762,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.238758,"completed_at":"2026-09-13T13:06:46.397Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011548,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2103gitlab # [ 430.046067] gitaly[919]: time="2026-09-13T13:06:46.933Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE17D9ZMQ15MPM1RGZQ4A6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:16.431" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:46.932" grpc.time_ms=1.308 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2104gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.26 seconds)2105gitlab # [ 430.110041] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1712106gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2107gitlab # [ 431.267997] gitaly[919]: time="2026-09-13T13:06:48.155Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE18KPHWJWFZ8QM4AHQ3T9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:18.053" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:48.154" grpc.time_ms=1.278 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2108gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.22 seconds)2109gitlab # [ 431.325676] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1582110gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2111gitlab # [ 432.483636] gitaly[919]: time="2026-09-13T13:06:49.371Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE19SPKXFPX12QR9K0ETYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:19.369" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:49.369" grpc.time_ms=1.22 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2112gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.22 seconds)2113gitlab # [ 432.545419] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1622114gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2115gitlab # [ 433.711453] gitaly[919]: time="2026-09-13T13:06:50.599Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1AZT4199S64587SNBM3C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:20.596" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:50.597" grpc.time_ms=1.422 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2116gitlab # [ 433.766557] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1632117gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.22 seconds)2118gitlab # [ 433.807595] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:50.694Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"f2ec2c0f5178ac236f937179","created_at":"2026-09-13T13:06:50.603Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06e694f4fa2a8ccc80c7dea6738141a7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.691Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-f2ec2c0f5178ac236f937179: start","job_status":"start","queue_duration_s":0.00368,"scheduling_latency_s":0.003745}2119gitlab # [ 434.014632] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:50.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"f2ec2c0f5178ac236f937179","created_at":"2026-09-13T13:06:50.603Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"06e694f4fa2a8ccc80c7dea6738141a7","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.691Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-f2ec2c0f5178ac236f937179: done: 0.207495 sec","job_status":"done","queue_duration_s":0.00368,"scheduling_latency_s":0.003745,"redis_calls":12,"redis_duration_s":0.068913,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.017754,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.046243,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004916,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0028077,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.207495,"completed_at":"2026-09-13T13:06:50.901Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021743,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2120gitlab # [ 434.113070] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:50.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b4a88285f791fc18ffe87a98","created_at":"2026-09-13T13:06:50.709Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12fe6ea0e290d1c3b4f93d5d7f9d77dd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.725Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b4a88285f791fc18ffe87a98: start","job_status":"start","queue_duration_s":0.186866,"scheduling_latency_s":0.18692}2121gitlab # [ 434.376637] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.264Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b4a88285f791fc18ffe87a98","created_at":"2026-09-13T13:06:50.709Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12fe6ea0e290d1c3b4f93d5d7f9d77dd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.725Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b4a88285f791fc18ffe87a98: done: 0.352346 sec","job_status":"done","queue_duration_s":0.186866,"scheduling_latency_s":0.18692,"redis_calls":11,"redis_duration_s":0.035502,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002573,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.023515,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.087,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.087,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.117462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00240136,"exclusive_lock_hold_duration_s":0.11329808,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.352346,"completed_at":"2026-09-13T13:06:51.264Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.099312,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2122gitlab # [ 434.474626] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a265dad3d78bb9b82c7b7647","created_at":"2026-09-13T13:06:50.773Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d72d495c6b7138e5fe5961f3299e1b25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.786Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a265dad3d78bb9b82c7b7647: start","job_status":"start","queue_duration_s":0.489641,"scheduling_latency_s":0.489697}2123gitlab # [ 434.520838] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.408Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"a265dad3d78bb9b82c7b7647","created_at":"2026-09-13T13:06:50.773Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d72d495c6b7138e5fe5961f3299e1b25","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.786Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-a265dad3d78bb9b82c7b7647: done: 0.132762 sec","job_status":"done","queue_duration_s":0.489641,"scheduling_latency_s":0.489697,"redis_calls":11,"redis_duration_s":0.019986,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004954,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01024,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004792,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047815,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00288044,"exclusive_lock_hold_duration_s":0.02384074,"duration_s":0.132762,"completed_at":"2026-09-13T13:06:51.408Z","load_balancing_strategy":"primary","db_duration_s":0.014732,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 434.615330] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"f0784162ff40d9047803b5c0","created_at":"2026-09-13T13:06:50.799Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9cdf44befeb580ae0df788497528eb1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.814Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-f0784162ff40d9047803b5c0: start","job_status":"start","queue_duration_s":0.604843,"scheduling_latency_s":0.604914}2125gitlab # [ 434.700830] nix-daemon[3738]: accepted connection from pid 4782, user nixbot2126gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2127gitlab # [ 434.708325] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"f0784162ff40d9047803b5c0","created_at":"2026-09-13T13:06:50.799Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9cdf44befeb580ae0df788497528eb1","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.814Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-f0784162ff40d9047803b5c0: done: 0.177046 sec","job_status":"done","queue_duration_s":0.604843,"scheduling_latency_s":0.604914,"redis_calls":9,"redis_duration_s":0.029814,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001494,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.024747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003573,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.177046,"completed_at":"2026-09-13T13:06:51.595Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012354,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2128gitlab # [ 434.815391] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1d9e9308f9272cc5505c396c","created_at":"2026-09-13T13:06:50.873Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2dda190d01764fd867f12d4a73b4b1d8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.909Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1d9e9308f9272cc5505c396c: start","job_status":"start","queue_duration_s":0.713497,"scheduling_latency_s":0.713547}2129gitlab # [ 434.837848] nix-daemon[3738]: accepted connection from pid 4745, user nixbot2130gitlab # [ 434.875225] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.763Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"1d9e9308f9272cc5505c396c","created_at":"2026-09-13T13:06:50.873Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2dda190d01764fd867f12d4a73b4b1d8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.909Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-1d9e9308f9272cc5505c396c: done: 0.14037 sec","job_status":"done","queue_duration_s":0.713497,"scheduling_latency_s":0.713547,"redis_calls":12,"redis_duration_s":0.027038,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004098,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01332,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00962,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.05532542,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.14037,"completed_at":"2026-09-13T13:06:51.762Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041599,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2131gitlab # [ 434.952939] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:51.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f97e63234f223b52aed13683","created_at":"2026-09-13T13:06:50.950Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.960Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f97e63234f223b52aed13683: start","job_status":"start","queue_duration_s":0.835748,"scheduling_latency_s":0.835805}2132gitlab # [ 434.968200] gitaly[919]: time="2026-09-13T13:06:51.836Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1C6JVP9EGSC484AGXZXV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:21.334" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:51.834" grpc.time_ms=1.607 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2133gitlab # [ 435.002586] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1602134gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.24 seconds)2135gitlab # [ 435.276372] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f97e63234f223b52aed13683","created_at":"2026-09-13T13:06:50.950Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.960Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f97e63234f223b52aed13683: done: 0.369053 sec","job_status":"done","queue_duration_s":0.835748,"scheduling_latency_s":0.835805,"redis_calls":70,"redis_duration_s":0.048583,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.008221,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001143,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.03309,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006129,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.061,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.19968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.369053,"completed_at":"2026-09-13T13:06:52.164Z","load_balancing_strategy":"primary","db_duration_s":0.06089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136gitlab # [ 435.369808] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"793fec97bbe8eb7290824859","created_at":"2026-09-13T13:06:50.974Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f52a595dd51f546aa069a26c210657a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.994Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-793fec97bbe8eb7290824859: start","job_status":"start","queue_duration_s":1.172518,"scheduling_latency_s":1.172554}2137gitlab # [ 435.388162] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.196Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"793fec97bbe8eb7290824859","created_at":"2026-09-13T13:06:50.974Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f52a595dd51f546aa069a26c210657a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:50.994Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-793fec97bbe8eb7290824859: done: 0.029686 sec","job_status":"done","queue_duration_s":1.172518,"scheduling_latency_s":1.172554,"redis_calls":11,"redis_duration_s":0.003077,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001152,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001382,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029686,"completed_at":"2026-09-13T13:06:52.196Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2138gitlab # [ 435.436319] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.198Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1e182bd87922ea16a2824fd3","created_at":"2026-09-13T13:06:51.039Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca97721acce87cc7255649be16476add","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.055Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1e182bd87922ea16a2824fd3: start","job_status":"start","queue_duration_s":1.142891,"scheduling_latency_s":1.142923}2139gitlab # [ 435.708532] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"1e182bd87922ea16a2824fd3","created_at":"2026-09-13T13:06:51.039Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ca97721acce87cc7255649be16476add","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.055Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-1e182bd87922ea16a2824fd3: done: 0.398377 sec","job_status":"done","queue_duration_s":1.142891,"scheduling_latency_s":1.142923,"redis_calls":564,"redis_duration_s":0.093502,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000992,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.091184,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.345963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.398377,"completed_at":"2026-09-13T13:06:52.596Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000993,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab # [ 435.804284] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.598Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"8f4789eff47982bcc16e0403","created_at":"2026-09-13T13:06:51.070Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"316453f1a8d3f1ddf571cb6624d28d8a","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.089Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-8f4789eff47982bcc16e0403: start","job_status":"start","queue_duration_s":1.509605,"scheduling_latency_s":1.509638}2141gitlab # [ 435.823463] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.621Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"8f4789eff47982bcc16e0403","created_at":"2026-09-13T13:06:51.070Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"316453f1a8d3f1ddf571cb6624d28d8a","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.089Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-8f4789eff47982bcc16e0403: done: 0.022421 sec","job_status":"done","queue_duration_s":1.509605,"scheduling_latency_s":1.509638,"redis_calls":11,"redis_duration_s":0.002874,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000529,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00054,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001284,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022421,"completed_at":"2026-09-13T13:06:52.621Z","load_balancing_strategy":"primary","db_duration_s":0.000934,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2142gitlab # [ 435.877082] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.622Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d78de3d2882489b93ca3923f","created_at":"2026-09-13T13:06:51.148Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a967b8ff7bcab50da3c048060f8bdf6d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.164Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d78de3d2882489b93ca3923f: start","job_status":"start","queue_duration_s":1.458565,"scheduling_latency_s":1.458591}2143gitlab # [ 435.892132] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.635Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2144gitlab # [ 435.895370] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d78de3d2882489b93ca3923f","created_at":"2026-09-13T13:06:51.148Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a967b8ff7bcab50da3c048060f8bdf6d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.164Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d78de3d2882489b93ca3923f: done: 0.019374 sec","job_status":"done","queue_duration_s":1.458565,"scheduling_latency_s":1.458591,"redis_calls":9,"redis_duration_s":0.002234,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001219,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019374,"completed_at":"2026-09-13T13:06:52.642Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000739,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 435.937770] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d14f923469d5e4a6424ab54d","created_at":"2026-09-13T13:06:51.182Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1774c7d1ac0126ca5eddcb8f49fd7350","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T13:06:51.200Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d14f923469d5e4a6424ab54d: start","job_status":"start","queue_duration_s":1.4434,"scheduling_latency_s":1.443427}2146gitlab # [ 435.951067] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.655Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2147gitlab # [ 435.954469] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"d14f923469d5e4a6424ab54d","created_at":"2026-09-13T13:06:51.182Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1774c7d1ac0126ca5eddcb8f49fd7350","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T13:06:51.200Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-d14f923469d5e4a6424ab54d: done: 0.018855 sec","job_status":"done","queue_duration_s":1.4434,"scheduling_latency_s":1.443427,"redis_calls":9,"redis_duration_s":0.002188,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001207,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018855,"completed_at":"2026-09-13T13:06:52.662Z","load_balancing_strategy":"primary","db_duration_s":0.000576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2148gitlab # [ 435.995241] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b5c4045b3d98588eb8a358ae","created_at":"2026-09-13T13:06:51.229Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d80a11ef02eb646e5e93a6bed029493d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.257Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b5c4045b3d98588eb8a358ae: start","job_status":"start","queue_duration_s":1.406851,"scheduling_latency_s":1.406877}2149gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2150gitlab # [ 436.008966] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"b5c4045b3d98588eb8a358ae","created_at":"2026-09-13T13:06:51.229Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d80a11ef02eb646e5e93a6bed029493d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.257Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-b5c4045b3d98588eb8a358ae: done: 0.036681 sec","job_status":"done","queue_duration_s":1.406851,"scheduling_latency_s":1.406877,"redis_calls":18,"redis_duration_s":0.004309,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000957,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000403,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002212,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000737,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02485,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036681,"completed_at":"2026-09-13T13:06:52.700Z","load_balancing_strategy":"primary","db_duration_s":0.003333,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151gitlab # [ 436.051496] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.702Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e171ad042444e3309ab400e9","created_at":"2026-09-13T13:06:51.271Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a08c0ce3f3ea7e6396995b8c8a06f18e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T13:06:51.288Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e171ad042444e3309ab400e9: start","job_status":"start","queue_duration_s":1.41449,"scheduling_latency_s":1.414517}2152gitlab # [ 436.064797] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.851Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"e171ad042444e3309ab400e9","created_at":"2026-09-13T13:06:51.271Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a08c0ce3f3ea7e6396995b8c8a06f18e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T13:06:51.288Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-e171ad042444e3309ab400e9: done: 0.148768 sec","job_status":"done","queue_duration_s":1.41449,"scheduling_latency_s":1.414517,"redis_calls":9,"redis_duration_s":0.002526,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000514,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001486,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000526,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.148768,"completed_at":"2026-09-13T13:06:52.851Z","load_balancing_strategy":"primary","db_duration_s":0.006402,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2153gitlab # [ 436.105315] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.853Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ad8d2d5dcc7848aecf322ec0","created_at":"2026-09-13T13:06:51.298Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a43b51141073fabf3d0918563db471c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.325Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ad8d2d5dcc7848aecf322ec0: start","job_status":"start","queue_duration_s":1.527953,"scheduling_latency_s":1.527977}2154gitlab # [ 436.119449] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.882Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ad8d2d5dcc7848aecf322ec0","created_at":"2026-09-13T13:06:51.298Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a43b51141073fabf3d0918563db471c","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.325Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ad8d2d5dcc7848aecf322ec0: done: 0.029877 sec","job_status":"done","queue_duration_s":1.527953,"scheduling_latency_s":1.527977,"redis_calls":9,"redis_duration_s":0.002711,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001493,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000592,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029877,"completed_at":"2026-09-13T13:06:52.882Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2155gitlab # [ 436.161195] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.884Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"499a5f5828cf9325c6c517ce","created_at":"2026-09-13T13:06:51.346Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e16d67bcb1a3f58083125bc02ad6259d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T13:06:51.369Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-499a5f5828cf9325c6c517ce: start","job_status":"start","queue_duration_s":1.515308,"scheduling_latency_s":1.515335}2156gitlab # [ 436.174762] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.902Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2157gitlab # [ 436.178525] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.911Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"499a5f5828cf9325c6c517ce","created_at":"2026-09-13T13:06:51.346Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e16d67bcb1a3f58083125bc02ad6259d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T13:06:51.369Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-499a5f5828cf9325c6c517ce: done: 0.026952 sec","job_status":"done","queue_duration_s":1.515308,"scheduling_latency_s":1.515335,"redis_calls":9,"redis_duration_s":0.007026,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000563,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003383,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012213,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026952,"completed_at":"2026-09-13T13:06:52.911Z","load_balancing_strategy":"primary","db_duration_s":0.0009,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2158gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.21 seconds)2159gitlab # [ 436.219313] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.913Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"254ac3033582f6f343461654","created_at":"2026-09-13T13:06:51.389Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99b270b36d36747c17e96a7387b96f8f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.416Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-254ac3033582f6f343461654: start","job_status":"start","queue_duration_s":1.497308,"scheduling_latency_s":1.497332}2160gitlab # [ 436.234960] gitaly[919]: time="2026-09-13T13:06:53.061Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1DDJRPBA528AHAE89KXJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:23.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:53.060" grpc.time_ms=0.852 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2161gitlab # [ 436.247942] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1172162gitlab # [ 436.250716] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"254ac3033582f6f343461654","created_at":"2026-09-13T13:06:51.389Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"99b270b36d36747c17e96a7387b96f8f","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.416Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-254ac3033582f6f343461654: done: 0.032239 sec","job_status":"done","queue_duration_s":1.497308,"scheduling_latency_s":1.497332,"redis_calls":11,"redis_duration_s":0.002868,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000675,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006101,"exclusive_lock_hold_duration_s":0.0098549,"duration_s":0.032239,"completed_at":"2026-09-13T13:06:52.945Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007586,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2163gitlab # [ 436.292193] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6e92c1f9e370ca58d54cedfe","created_at":"2026-09-13T13:06:51.433Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a57ffefcb3ea32c4713ff4e1f249836f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.445Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6e92c1f9e370ca58d54cedfe: start","job_status":"start","queue_duration_s":1.501775,"scheduling_latency_s":1.501802}2164gitlab # [ 436.306505] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.968Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6e92c1f9e370ca58d54cedfe","created_at":"2026-09-13T13:06:51.433Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a57ffefcb3ea32c4713ff4e1f249836f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.445Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6e92c1f9e370ca58d54cedfe: done: 0.021423 sec","job_status":"done","queue_duration_s":1.501775,"scheduling_latency_s":1.501802,"redis_calls":9,"redis_duration_s":0.001821,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000405,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000443,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021423,"completed_at":"2026-09-13T13:06:52.968Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2165gitlab # [ 436.348082] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:52.970Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c57b288ad4f6ebf97adec9ac","created_at":"2026-09-13T13:06:51.480Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d953e5d31df65a4faa765e1364529a8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.529Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c57b288ad4f6ebf97adec9ac: start","job_status":"start","queue_duration_s":1.440999,"scheduling_latency_s":1.441029}2166gitlab # [ 436.362147] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.004Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2167gitlab # [ 436.365879] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"c57b288ad4f6ebf97adec9ac","created_at":"2026-09-13T13:06:51.480Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d953e5d31df65a4faa765e1364529a8c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.529Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-c57b288ad4f6ebf97adec9ac: done: 0.047662 sec","job_status":"done","queue_duration_s":1.440999,"scheduling_latency_s":1.441029,"redis_calls":9,"redis_duration_s":0.012248,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001656,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000708,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047662,"completed_at":"2026-09-13T13:06:53.018Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 436.407834] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.020Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"55e775cae3111dbecff1ae7c","created_at":"2026-09-13T13:06:51.571Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bea6de47d0f697cb0f815c8f507d97b0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.603Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-55e775cae3111dbecff1ae7c: start","job_status":"start","queue_duration_s":1.416403,"scheduling_latency_s":1.416434}2169gitlab # [ 436.421481] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.076Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"55e775cae3111dbecff1ae7c","created_at":"2026-09-13T13:06:51.571Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bea6de47d0f697cb0f815c8f507d97b0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.603Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-55e775cae3111dbecff1ae7c: done: 0.056382 sec","job_status":"done","queue_duration_s":1.416403,"scheduling_latency_s":1.416434,"redis_calls":9,"redis_duration_s":0.003188,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00065,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.056382,"completed_at":"2026-09-13T13:06:53.076Z","load_balancing_strategy":"primary","db_duration_s":0.019498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170gitlab # [ 436.463270] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"314d3ed498b3d11fcdf5cc75","created_at":"2026-09-13T13:06:51.618Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7957177735e9752dc22a0a1ec6794379","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.651Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-314d3ed498b3d11fcdf5cc75: start","job_status":"start","queue_duration_s":1.427918,"scheduling_latency_s":1.427956}2171gitlab # [ 436.477502] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.186Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"314d3ed498b3d11fcdf5cc75","created_at":"2026-09-13T13:06:51.618Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7957177735e9752dc22a0a1ec6794379","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.651Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-314d3ed498b3d11fcdf5cc75: done: 0.10665 sec","job_status":"done","queue_duration_s":1.427918,"scheduling_latency_s":1.427956,"redis_calls":13,"redis_duration_s":0.004914,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001051,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001874,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00119934,"exclusive_lock_hold_duration_s":0.05546752,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.10665,"completed_at":"2026-09-13T13:06:53.186Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033986,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 436.520813] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.189Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ad4509e1f9c2f84c6e75763c","created_at":"2026-09-13T13:06:51.699Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f1ef6d3290b55866dfa765095e4ac46","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.722Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ad4509e1f9c2f84c6e75763c: start","job_status":"start","queue_duration_s":1.467096,"scheduling_latency_s":1.467131}2173gitlab # [ 436.534739] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"ad4509e1f9c2f84c6e75763c","created_at":"2026-09-13T13:06:51.699Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6f1ef6d3290b55866dfa765095e4ac46","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.722Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-ad4509e1f9c2f84c6e75763c: done: 0.040625 sec","job_status":"done","queue_duration_s":1.467096,"scheduling_latency_s":1.467131,"redis_calls":9,"redis_duration_s":0.003159,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000627,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001806,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000726,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040625,"completed_at":"2026-09-13T13:06:53.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2174gitlab # [ 436.575416] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.232Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7c4d17684bfbf32d27030eea","created_at":"2026-09-13T13:06:51.739Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3e8de0d9e453431628736ab63df680e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.758Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7c4d17684bfbf32d27030eea: start","job_status":"start","queue_duration_s":1.473691,"scheduling_latency_s":1.473719}2175gitlab # [ 436.589323] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.254Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2176gitlab # [ 436.592441] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7c4d17684bfbf32d27030eea","created_at":"2026-09-13T13:06:51.739Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3e8de0d9e453431628736ab63df680e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.758Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7c4d17684bfbf32d27030eea: done: 0.044123 sec","job_status":"done","queue_duration_s":1.473691,"scheduling_latency_s":1.473719,"redis_calls":9,"redis_duration_s":0.003699,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000872,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002058,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000769,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026158,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044123,"completed_at":"2026-09-13T13:06:53.276Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001168,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2177gitlab # [ 436.633240] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5278f092facffaeacf562dce","created_at":"2026-09-13T13:06:51.794Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"330a2eadb7c4f8b45e0b5bde639e4783","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.810Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5278f092facffaeacf562dce: start","job_status":"start","queue_duration_s":1.469446,"scheduling_latency_s":1.469489}2178gitlab # [ 436.753220] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.641Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"5278f092facffaeacf562dce","created_at":"2026-09-13T13:06:51.794Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"330a2eadb7c4f8b45e0b5bde639e4783","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.810Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-5278f092facffaeacf562dce: done: 0.362379 sec","job_status":"done","queue_duration_s":1.469446,"scheduling_latency_s":1.469489,"redis_calls":13,"redis_duration_s":0.00452,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001469,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00159,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001461,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.055,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.055,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.284894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00107616,"exclusive_lock_hold_duration_s":0.31855264,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.362379,"completed_at":"2026-09-13T13:06:53.641Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.082768,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2179gitlab # [ 436.842890] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b2e4fe5f70cb57ee8e8f8a74","created_at":"2026-09-13T13:06:51.816Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acb49bbdbaa535b08a4bcca9a6b0c551","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.876Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b2e4fe5f70cb57ee8e8f8a74: start","job_status":"start","queue_duration_s":1.766543,"scheduling_latency_s":1.766561}2180gitlab # [ 436.885649] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"b2e4fe5f70cb57ee8e8f8a74","created_at":"2026-09-13T13:06:51.816Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"acb49bbdbaa535b08a4bcca9a6b0c551","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.876Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-b2e4fe5f70cb57ee8e8f8a74: done: 0.024272 sec","job_status":"done","queue_duration_s":1.766543,"scheduling_latency_s":1.766561,"redis_calls":11,"redis_duration_s":0.00301,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001268,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057646,"exclusive_lock_hold_duration_s":0.00357404,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.024272,"completed_at":"2026-09-13T13:06:53.667Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2181gitlab # [ 436.957355] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.669Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3554e5b6e21aa6f1c1d37020","created_at":"2026-09-13T13:06:51.889Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b162a481ec96ffe263173212af0aa9ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.912Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3554e5b6e21aa6f1c1d37020: start","job_status":"start","queue_duration_s":1.756121,"scheduling_latency_s":1.756137}2182gitlab # [ 436.997277] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.680Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2183gitlab # [ 437.004823] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3554e5b6e21aa6f1c1d37020","created_at":"2026-09-13T13:06:51.889Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-49bfe2a2f0c14255","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b162a481ec96ffe263173212af0aa9ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T13:06:51.912Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3554e5b6e21aa6f1c1d37020: done: 0.018259 sec","job_status":"done","queue_duration_s":1.756121,"scheduling_latency_s":1.756137,"redis_calls":9,"redis_duration_s":0.002075,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000455,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000453,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018259,"completed_at":"2026-09-13T13:06:53.687Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2184gitlab # [ 437.065621] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6640322d81b55a465b618e2e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6640322d81b55a465b618e2e: start","job_status":"start","queue_duration_s":1.536604,"scheduling_latency_s":1.536622}2185gitlab # [ 437.079588] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6640322d81b55a465b618e2e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6640322d81b55a465b618e2e: done: 0.022222 sec","job_status":"done","queue_duration_s":1.536604,"scheduling_latency_s":1.536622,"redis_calls":15,"redis_duration_s":0.003464,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000467,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022222,"completed_at":"2026-09-13T13:06:53.710Z","load_balancing_strategy":"primary","db_duration_s":0.000483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2186gitlab # [ 437.120177] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9da4926dae36a3fb4cc1c57d","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9da4926dae36a3fb4cc1c57d: start","job_status":"start","queue_duration_s":1.560212,"scheduling_latency_s":1.560229}2187gitlab # [ 437.132347] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9da4926dae36a3fb4cc1c57d","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9da4926dae36a3fb4cc1c57d: done: 0.015047 sec","job_status":"done","queue_duration_s":1.560212,"scheduling_latency_s":1.560229,"redis_calls":13,"redis_duration_s":0.002886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000989,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015047,"completed_at":"2026-09-13T13:06:53.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188gitlab # [ 437.171444] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.728Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b4d318b71532c97c81da5e3","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b4d318b71532c97c81da5e3: start","job_status":"start","queue_duration_s":1.576664,"scheduling_latency_s":1.57668}2189gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2190gitlab # [ 437.208476] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.743Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b4d318b71532c97c81da5e3","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b4d318b71532c97c81da5e3: done: 0.014485 sec","job_status":"done","queue_duration_s":1.576664,"scheduling_latency_s":1.57668,"redis_calls":13,"redis_duration_s":0.001915,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000675,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014485,"completed_at":"2026-09-13T13:06:53.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2191gitlab # [ 437.282559] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b89c1a0be130a32d163b5d","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b89c1a0be130a32d163b5d: start","job_status":"start","queue_duration_s":1.592315,"scheduling_latency_s":1.592331}2192gitlab # [ 437.301454] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6b89c1a0be130a32d163b5d","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6b89c1a0be130a32d163b5d: done: 0.015039 sec","job_status":"done","queue_duration_s":1.592315,"scheduling_latency_s":1.592331,"redis_calls":13,"redis_duration_s":0.002321,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001636,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015039,"completed_at":"2026-09-13T13:06:53.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2193gitlab # [ 437.345953] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f16616f531681381c7760cf","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f16616f531681381c7760cf: start","job_status":"start","queue_duration_s":1.608587,"scheduling_latency_s":1.608604}2194gitlab # [ 437.358112] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6f16616f531681381c7760cf","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6f16616f531681381c7760cf: done: 0.014854 sec","job_status":"done","queue_duration_s":1.608587,"scheduling_latency_s":1.608604,"redis_calls":13,"redis_duration_s":0.002307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00151,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014854,"completed_at":"2026-09-13T13:06:53.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2195gitlab # [ 437.396833] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.776Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5eaa3634647c883b3412fb4","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5eaa3634647c883b3412fb4: start","job_status":"start","queue_duration_s":1.624751,"scheduling_latency_s":1.624767}2196gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.21 seconds)2197gitlab # [ 437.409083] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.791Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5eaa3634647c883b3412fb4","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5eaa3634647c883b3412fb4: done: 0.015038 sec","job_status":"done","queue_duration_s":1.624751,"scheduling_latency_s":1.624767,"redis_calls":13,"redis_duration_s":0.002239,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015038,"completed_at":"2026-09-13T13:06:53.791Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2198gitlab # [ 437.447936] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03f09c10cd5f6b1bd142db20","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03f09c10cd5f6b1bd142db20: start","job_status":"start","queue_duration_s":1.641068,"scheduling_latency_s":1.641084}2199gitlab # [ 437.460806] gitaly[919]: time="2026-09-13T13:06:54.263Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1EKC3AXHW0CRJ8D29RSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:24.061" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:54.262" grpc.time_ms=1.035 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2200gitlab # [ 437.474972] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1282201gitlab # [ 437.477691] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"03f09c10cd5f6b1bd142db20","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-03f09c10cd5f6b1bd142db20: done: 0.014968 sec","job_status":"done","queue_duration_s":1.641068,"scheduling_latency_s":1.641084,"redis_calls":13,"redis_duration_s":0.002496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001595,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014968,"completed_at":"2026-09-13T13:06:53.808Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 437.516544] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11a2ce78e3a0200ea718e8be","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11a2ce78e3a0200ea718e8be: start","job_status":"start","queue_duration_s":1.657483,"scheduling_latency_s":1.6575}2203gitlab # [ 437.528659] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11a2ce78e3a0200ea718e8be","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11a2ce78e3a0200ea718e8be: done: 0.01663 sec","job_status":"done","queue_duration_s":1.657483,"scheduling_latency_s":1.6575,"redis_calls":13,"redis_duration_s":0.003268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000978,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01663,"completed_at":"2026-09-13T13:06:53.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2204gitlab # [ 437.567064] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67bd7756c9704967c2a23c20","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67bd7756c9704967c2a23c20: start","job_status":"start","queue_duration_s":1.675871,"scheduling_latency_s":1.675889}2205gitlab # [ 437.579699] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67bd7756c9704967c2a23c20","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67bd7756c9704967c2a23c20: done: 0.015533 sec","job_status":"done","queue_duration_s":1.675871,"scheduling_latency_s":1.675889,"redis_calls":13,"redis_duration_s":0.002827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001807,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015533,"completed_at":"2026-09-13T13:06:53.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206gitlab # [ 437.618987] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"156aa2d7aa97df3b5b5bb13f","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-156aa2d7aa97df3b5b5bb13f: start","job_status":"start","queue_duration_s":1.692789,"scheduling_latency_s":1.692806}2207gitlab # [ 437.631547] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"156aa2d7aa97df3b5b5bb13f","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-156aa2d7aa97df3b5b5bb13f: done: 0.017627 sec","job_status":"done","queue_duration_s":1.692789,"scheduling_latency_s":1.692806,"redis_calls":13,"redis_duration_s":0.004602,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002236,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002366,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017627,"completed_at":"2026-09-13T13:06:53.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208gitlab # [ 437.671025] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.864Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92a2942c5707effe9594edf9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92a2942c5707effe9594edf9: start","job_status":"start","queue_duration_s":1.712099,"scheduling_latency_s":1.712116}2209gitlab # [ 437.685104] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92a2942c5707effe9594edf9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92a2942c5707effe9594edf9: done: 0.02614 sec","job_status":"done","queue_duration_s":1.712099,"scheduling_latency_s":1.712116,"redis_calls":13,"redis_duration_s":0.003827,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002686,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02614,"completed_at":"2026-09-13T13:06:53.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2210gitlab # [ 437.722998] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"202001294571f56d89d44421","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-202001294571f56d89d44421: start","job_status":"start","queue_duration_s":1.74083,"scheduling_latency_s":1.740869}2211gitlab # [ 437.735240] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"202001294571f56d89d44421","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-202001294571f56d89d44421: done: 0.036564 sec","job_status":"done","queue_duration_s":1.74083,"scheduling_latency_s":1.740869,"redis_calls":13,"redis_duration_s":0.003651,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002406,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.036564,"completed_at":"2026-09-13T13:06:53.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab # [ 437.773682] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9b0a57aaa27e07e54721dab","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9b0a57aaa27e07e54721dab: start","job_status":"start","queue_duration_s":1.780816,"scheduling_latency_s":1.780862}2213gitlab # [ 437.785904] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f9b0a57aaa27e07e54721dab","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f9b0a57aaa27e07e54721dab: done: 0.041103 sec","job_status":"done","queue_duration_s":1.780816,"scheduling_latency_s":1.780862,"redis_calls":13,"redis_duration_s":0.005713,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003821,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.041103,"completed_at":"2026-09-13T13:06:53.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214gitlab # [ 437.823836] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:53.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1a92a3d1f0af88b7ec52f3f","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1a92a3d1f0af88b7ec52f3f: start","job_status":"start","queue_duration_s":1.825067,"scheduling_latency_s":1.825115}2215gitlab # [ 437.836190] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1a92a3d1f0af88b7ec52f3f","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1a92a3d1f0af88b7ec52f3f: done: 0.038189 sec","job_status":"done","queue_duration_s":1.825067,"scheduling_latency_s":1.825115,"redis_calls":13,"redis_duration_s":0.004654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038189,"completed_at":"2026-09-13T13:06:54.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 437.874946] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e51dbead8d67774bf73c590","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e51dbead8d67774bf73c590: start","job_status":"start","queue_duration_s":1.865902,"scheduling_latency_s":1.865943}2217gitlab # [ 437.888542] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e51dbead8d67774bf73c590","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e51dbead8d67774bf73c590: done: 0.028435 sec","job_status":"done","queue_duration_s":1.865902,"scheduling_latency_s":1.865943,"redis_calls":13,"redis_duration_s":0.0036049999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00123,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002375,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028435,"completed_at":"2026-09-13T13:06:54.046Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2218gitlab # [ 437.926676] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c4b126659202fd3a3c6763d","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c4b126659202fd3a3c6763d: start","job_status":"start","queue_duration_s":1.896177,"scheduling_latency_s":1.896202}2219gitlab # [ 437.939022] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c4b126659202fd3a3c6763d","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c4b126659202fd3a3c6763d: done: 0.023981 sec","job_status":"done","queue_duration_s":1.896177,"scheduling_latency_s":1.896202,"redis_calls":13,"redis_duration_s":0.003522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001173,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002349,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023981,"completed_at":"2026-09-13T13:06:54.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220gitlab # [ 437.977157] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caa8abd541e2c0824c8757dc","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caa8abd541e2c0824c8757dc: start","job_status":"start","queue_duration_s":1.922055,"scheduling_latency_s":1.922078}2221gitlab # [ 437.988983] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"caa8abd541e2c0824c8757dc","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-caa8abd541e2c0824c8757dc: done: 0.02275 sec","job_status":"done","queue_duration_s":1.922055,"scheduling_latency_s":1.922078,"redis_calls":13,"redis_duration_s":0.0038269999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00125,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002577,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02275,"completed_at":"2026-09-13T13:06:54.096Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2222gitlab # [ 438.027171] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4639018de7eea0f6a2bab27","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4639018de7eea0f6a2bab27: start","job_status":"start","queue_duration_s":1.946543,"scheduling_latency_s":1.946562}2223gitlab # [ 438.039459] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4639018de7eea0f6a2bab27","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4639018de7eea0f6a2bab27: done: 0.018865 sec","job_status":"done","queue_duration_s":1.946543,"scheduling_latency_s":1.946562,"redis_calls":13,"redis_duration_s":0.003116,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001127,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018865,"completed_at":"2026-09-13T13:06:54.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 438.078617] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b873507fec773eb6589ba6fb","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b873507fec773eb6589ba6fb: start","job_status":"start","queue_duration_s":1.966877,"scheduling_latency_s":1.966908}2225gitlab # [ 438.091310] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b873507fec773eb6589ba6fb","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b873507fec773eb6589ba6fb: done: 0.017137 sec","job_status":"done","queue_duration_s":1.966877,"scheduling_latency_s":1.966908,"redis_calls":13,"redis_duration_s":0.0029449999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017137,"completed_at":"2026-09-13T13:06:54.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2226gitlab # [ 438.129468] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0931c2f9cd105c90506557a5","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0931c2f9cd105c90506557a5: start","job_status":"start","queue_duration_s":1.985696,"scheduling_latency_s":1.985714}2227gitlab # [ 438.141630] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0931c2f9cd105c90506557a5","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0931c2f9cd105c90506557a5: done: 0.017473 sec","job_status":"done","queue_duration_s":1.985696,"scheduling_latency_s":1.985714,"redis_calls":13,"redis_duration_s":0.00285,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001083,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001767,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017473,"completed_at":"2026-09-13T13:06:54.155Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2228gitlab # [ 438.179671] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56af9d08fc4794ee38c0610","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56af9d08fc4794ee38c0610: start","job_status":"start","queue_duration_s":2.004768,"scheduling_latency_s":2.004785}2229gitlab # [ 438.192032] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b56af9d08fc4794ee38c0610","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b56af9d08fc4794ee38c0610: done: 0.034835 sec","job_status":"done","queue_duration_s":2.004768,"scheduling_latency_s":2.004785,"redis_calls":13,"redis_duration_s":0.003803,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001246,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002557,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025993,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034835,"completed_at":"2026-09-13T13:06:54.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2230gitlab # [ 438.229602] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"003c0c54d39d3a70ff862522","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-003c0c54d39d3a70ff862522: start","job_status":"start","queue_duration_s":2.042233,"scheduling_latency_s":2.042269}2231gitlab # [ 438.241457] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"003c0c54d39d3a70ff862522","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-003c0c54d39d3a70ff862522: done: 0.026204 sec","job_status":"done","queue_duration_s":2.042233,"scheduling_latency_s":2.042269,"redis_calls":13,"redis_duration_s":0.002783,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000888,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026204,"completed_at":"2026-09-13T13:06:54.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2232gitlab # [ 438.279602] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a93c9c650fcda28ddb84147f","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a93c9c650fcda28ddb84147f: start","job_status":"start","queue_duration_s":2.070572,"scheduling_latency_s":2.070602}2233gitlab # [ 438.292751] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.243Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a93c9c650fcda28ddb84147f","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a93c9c650fcda28ddb84147f: done: 0.020772 sec","job_status":"done","queue_duration_s":2.070572,"scheduling_latency_s":2.070602,"redis_calls":13,"redis_duration_s":0.002351,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000725,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001626,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020772,"completed_at":"2026-09-13T13:06:54.243Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2234gitlab # [ 438.330446] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4608082a2db8413f4f7db74e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4608082a2db8413f4f7db74e: start","job_status":"start","queue_duration_s":2.092797,"scheduling_latency_s":2.09282}2235gitlab # [ 438.343237] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.265Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4608082a2db8413f4f7db74e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4608082a2db8413f4f7db74e: done: 0.020898 sec","job_status":"done","queue_duration_s":2.092797,"scheduling_latency_s":2.09282,"redis_calls":13,"redis_duration_s":0.002239,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001476,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020898,"completed_at":"2026-09-13T13:06:54.265Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236gitlab # [ 438.382133] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"564e6f0c1ab36a290cc63f2e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-564e6f0c1ab36a290cc63f2e: start","job_status":"start","queue_duration_s":2.115365,"scheduling_latency_s":2.115392}2237gitlab # [ 438.394586] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"564e6f0c1ab36a290cc63f2e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-564e6f0c1ab36a290cc63f2e: done: 0.027401 sec","job_status":"done","queue_duration_s":2.115365,"scheduling_latency_s":2.115392,"redis_calls":13,"redis_duration_s":0.003451,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002207,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027401,"completed_at":"2026-09-13T13:06:54.294Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2238gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2239gitlab # [ 438.433249] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a08b31c7e092dbd5cf0f9d1","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a08b31c7e092dbd5cf0f9d1: start","job_status":"start","queue_duration_s":2.144841,"scheduling_latency_s":2.144872}2240gitlab # [ 438.445402] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.324Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a08b31c7e092dbd5cf0f9d1","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a08b31c7e092dbd5cf0f9d1: done: 0.027726 sec","job_status":"done","queue_duration_s":2.144841,"scheduling_latency_s":2.144872,"redis_calls":13,"redis_duration_s":0.003239,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002069,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027726,"completed_at":"2026-09-13T13:06:54.324Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2241gitlab # [ 438.481166] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c678a61336bb9d1ceab5b301","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c678a61336bb9d1ceab5b301: start","job_status":"start","queue_duration_s":2.17426,"scheduling_latency_s":2.174286}2242gitlab # [ 438.493311] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c678a61336bb9d1ceab5b301","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c678a61336bb9d1ceab5b301: done: 0.018597 sec","job_status":"done","queue_duration_s":2.17426,"scheduling_latency_s":2.174286,"redis_calls":13,"redis_duration_s":0.002393,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000848,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001545,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018597,"completed_at":"2026-09-13T13:06:54.344Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2243gitlab # [ 438.532996] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0379ac246921d6f50c923dab","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0379ac246921d6f50c923dab: start","job_status":"start","queue_duration_s":2.194457,"scheduling_latency_s":2.194479}2244gitlab # [ 438.545310] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0379ac246921d6f50c923dab","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0379ac246921d6f50c923dab: done: 0.018093 sec","job_status":"done","queue_duration_s":2.194457,"scheduling_latency_s":2.194479,"redis_calls":13,"redis_duration_s":0.003022,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010763,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018093,"completed_at":"2026-09-13T13:06:54.364Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2245gitlab # [ 438.583445] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.366Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce24bf49a4a8cd583470b53e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce24bf49a4a8cd583470b53e: start","job_status":"start","queue_duration_s":2.214195,"scheduling_latency_s":2.214219}2246gitlab # [ 438.595949] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce24bf49a4a8cd583470b53e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce24bf49a4a8cd583470b53e: done: 0.018056 sec","job_status":"done","queue_duration_s":2.214195,"scheduling_latency_s":2.214219,"redis_calls":13,"redis_duration_s":0.003371,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001356,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002015,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018056,"completed_at":"2026-09-13T13:06:54.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2247gitlab # [ 438.634339] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed78102d408b52bd61e475fc","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed78102d408b52bd61e475fc: start","job_status":"start","queue_duration_s":2.23412,"scheduling_latency_s":2.234146}2248gitlab # [ 438.646484] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed78102d408b52bd61e475fc","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed78102d408b52bd61e475fc: done: 0.017949 sec","job_status":"done","queue_duration_s":2.23412,"scheduling_latency_s":2.234146,"redis_calls":13,"redis_duration_s":0.002921,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017949,"completed_at":"2026-09-13T13:06:54.404Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2249gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.26 seconds)2250gitlab # [ 438.684842] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19643bb0c7631d23ecdf0a31","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19643bb0c7631d23ecdf0a31: start","job_status":"start","queue_duration_s":2.253852,"scheduling_latency_s":2.253875}2251gitlab # [ 438.697092] gitaly[919]: time="2026-09-13T13:06:55.517Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FSWQT9VC40AYZZH603W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:25.015" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:55.515" grpc.time_ms=1.22 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2252gitlab # [ 438.711054] nix-daemon[3738]: accepted connection from pid 4745, user nixbot2253gitlab # [ 438.712031] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19643bb0c7631d23ecdf0a31","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19643bb0c7631d23ecdf0a31: done: 0.016296 sec","job_status":"done","queue_duration_s":2.253852,"scheduling_latency_s":2.253875,"redis_calls":13,"redis_duration_s":0.0027660000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016296,"completed_at":"2026-09-13T13:06:54.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2254gitlab # [ 438.750350] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e193bc7c8f8836cb475c341e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e193bc7c8f8836cb475c341e: start","job_status":"start","queue_duration_s":2.271977,"scheduling_latency_s":2.272001}2255gitlab # [ 438.762778] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1452256gitlab # [ 438.765381] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e193bc7c8f8836cb475c341e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e193bc7c8f8836cb475c341e: done: 0.016718 sec","job_status":"done","queue_duration_s":2.271977,"scheduling_latency_s":2.272001,"redis_calls":13,"redis_duration_s":0.002823,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016718,"completed_at":"2026-09-13T13:06:54.440Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2257gitlab # [ 438.803971] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af824c13257789db242d73f9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af824c13257789db242d73f9: start","job_status":"start","queue_duration_s":2.290517,"scheduling_latency_s":2.290541}2258gitlab # [ 438.816331] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af824c13257789db242d73f9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af824c13257789db242d73f9: done: 0.018306 sec","job_status":"done","queue_duration_s":2.290517,"scheduling_latency_s":2.290541,"redis_calls":13,"redis_duration_s":0.003159,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002179,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018306,"completed_at":"2026-09-13T13:06:54.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2259gitlab # [ 438.854749] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.463Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da3fcc18ea88539330eadbbb","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da3fcc18ea88539330eadbbb: start","job_status":"start","queue_duration_s":2.311366,"scheduling_latency_s":2.311393}2260gitlab # [ 438.867076] gitaly[919]: time="2026-09-13T13:06:55.700Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:25.197" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:55.698" grpc.time_ms=1.194 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2261gitlab # [ 438.879878] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.482Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da3fcc18ea88539330eadbbb","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da3fcc18ea88539330eadbbb: done: 0.019278 sec","job_status":"done","queue_duration_s":2.311366,"scheduling_latency_s":2.311393,"redis_calls":13,"redis_duration_s":0.002989,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002042,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019278,"completed_at":"2026-09-13T13:06:54.482Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262gitlab # [ 438.919445] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a47e1597e9326ce4613c86e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a47e1597e9326ce4613c86e: start","job_status":"start","queue_duration_s":2.332235,"scheduling_latency_s":2.33226}2263gitlab # [ 438.931779] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a47e1597e9326ce4613c86e","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a47e1597e9326ce4613c86e: done: 0.018898 sec","job_status":"done","queue_duration_s":2.332235,"scheduling_latency_s":2.33226,"redis_calls":13,"redis_duration_s":0.002956,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018898,"completed_at":"2026-09-13T13:06:54.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2264gitlab # [ 438.970341] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d98a465573b3a54909af874a","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d98a465573b3a54909af874a: start","job_status":"start","queue_duration_s":2.352764,"scheduling_latency_s":2.352788}2265gitlab # [ 438.982859] gitaly[919]: time="2026-09-13T13:06:55.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:07:05.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:55.759" grpc.time_ms=17.601 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2266gitlab # [ 438.996324] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d98a465573b3a54909af874a","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d98a465573b3a54909af874a: done: 0.018798 sec","job_status":"done","queue_duration_s":2.352764,"scheduling_latency_s":2.352788,"redis_calls":13,"redis_duration_s":0.003276,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002247,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018798,"completed_at":"2026-09-13T13:06:54.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2267gitlab # [ 439.034762] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.525Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c67ce919a2d27ef47c4230a","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c67ce919a2d27ef47c4230a: start","job_status":"start","queue_duration_s":2.373243,"scheduling_latency_s":2.373267}2268gitlab # [ 439.047421] nixbot[3565]: {"ts": "2026-09-13T13:06:55.873Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 \"HTTP/1.1 201 Created\""}2269gitlab # [ 439.050322] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:55 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2682270gitlab # [ 439.052804] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c67ce919a2d27ef47c4230a","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c67ce919a2d27ef47c4230a: done: 0.015981 sec","job_status":"done","queue_duration_s":2.373243,"scheduling_latency_s":2.373267,"redis_calls":13,"redis_duration_s":0.002954,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009807,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015981,"completed_at":"2026-09-13T13:06:54.541Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2271gitlab # [ 439.091339] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.542Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a87f943ebd72b62e2543b8d9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a87f943ebd72b62e2543b8d9: start","job_status":"start","queue_duration_s":2.390666,"scheduling_latency_s":2.390691}2272gitlab # [ 439.104047] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a87f943ebd72b62e2543b8d9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a87f943ebd72b62e2543b8d9: done: 0.016126 sec","job_status":"done","queue_duration_s":2.390666,"scheduling_latency_s":2.390691,"redis_calls":13,"redis_duration_s":0.003193,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016126,"completed_at":"2026-09-13T13:06:54.558Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2273gitlab # [ 439.143628] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.560Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cbe6c1fd5f26041886a70fd","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cbe6c1fd5f26041886a70fd: start","job_status":"start","queue_duration_s":2.408248,"scheduling_latency_s":2.408272}2274gitlab # [ 439.155907] gitaly[919]: time="2026-09-13T13:06:55.978Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1G86XR748JG63QTSGWBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:25.476" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:55.977" grpc.time_ms=1.103 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2275gitlab # [ 439.168341] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.576Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cbe6c1fd5f26041886a70fd","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cbe6c1fd5f26041886a70fd: done: 0.015919 sec","job_status":"done","queue_duration_s":2.408248,"scheduling_latency_s":2.408272,"redis_calls":13,"redis_duration_s":0.002787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015919,"completed_at":"2026-09-13T13:06:54.576Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2276gitlab # [ 439.206567] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e74d9460a2fc949f334b073","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e74d9460a2fc949f334b073: start","job_status":"start","queue_duration_s":2.425619,"scheduling_latency_s":2.425643}2277gitlab # [ 439.218612] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e74d9460a2fc949f334b073","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e74d9460a2fc949f334b073: done: 0.016786 sec","job_status":"done","queue_duration_s":2.425619,"scheduling_latency_s":2.425643,"redis_calls":13,"redis_duration_s":0.002952,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016786,"completed_at":"2026-09-13T13:06:54.594Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2278gitlab # [ 439.258140] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a9a73a52b3dc8ac2d46f8ae","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a9a73a52b3dc8ac2d46f8ae: start","job_status":"start","queue_duration_s":2.443865,"scheduling_latency_s":2.44389}2279gitlab # [ 439.270368] gitaly[919]: time="2026-09-13T13:06:56.067Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE1G86XR748JG63QTSGWBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:07:06.011" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.052" grpc.time_ms=14.674 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2280gitlab # [ 439.284487] gitaly[919]: time="2026-09-13T13:06:56.072Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:56.071" grpc.time_ms=0.787 pid=919 span.kind=server system=grpc2281gitlab # [ 439.294008] nixbot[3565]: {"ts": "2026-09-13T13:06:56.157Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 \"HTTP/1.1 201 Created\""}2282gitlab # [ 439.296856] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a9a73a52b3dc8ac2d46f8ae","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a9a73a52b3dc8ac2d46f8ae: done: 0.034745 sec","job_status":"done","queue_duration_s":2.443865,"scheduling_latency_s":2.44389,"redis_calls":13,"redis_duration_s":0.005738999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001878,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003861,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034745,"completed_at":"2026-09-13T13:06:54.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283gitlab # [ 439.337485] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d68ba368bfa37777f2e48753","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d68ba368bfa37777f2e48753: start","job_status":"start","queue_duration_s":2.481788,"scheduling_latency_s":2.481837}2284gitlab # [ 439.349841] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2762285gitlab # [ 439.352340] gitaly[919]: time="2026-09-13T13:06:56.108Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:56.107" grpc.time_ms=0.793 pid=919 span.kind=server system=grpc2286gitlab # [ 439.361488] gitaly[919]: time="2026-09-13T13:06:56.148Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T13:06:56.148" grpc.time_ms=0.496 pid=919 span.kind=server system=grpc2287gitlab # [ 439.371383] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d68ba368bfa37777f2e48753","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d68ba368bfa37777f2e48753: done: 0.035003 sec","job_status":"done","queue_duration_s":2.481788,"scheduling_latency_s":2.481837,"redis_calls":13,"redis_duration_s":0.004857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001446,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003411,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035003,"completed_at":"2026-09-13T13:06:54.668Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2288gitlab # [ 439.410336] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2faab242db60b192373373a9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2faab242db60b192373373a9: start","job_status":"start","queue_duration_s":2.519261,"scheduling_latency_s":2.519302}2289gitlab # [ 439.422804] gitaly[919]: time="2026-09-13T13:06:56.161Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:26.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.022" operations="[repack_refs]" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2290gitlab # [ 439.432628] gitaly[919]: time="2026-09-13T13:06:56.164Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:26.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.022" pid=919 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-13 13:05:05.54718676 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2291gitlab # [ 439.444237] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.701Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2faab242db60b192373373a9","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2faab242db60b192373373a9: done: 0.030338 sec","job_status":"done","queue_duration_s":2.519261,"scheduling_latency_s":2.519302,"redis_calls":13,"redis_duration_s":0.004191,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00278,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030338,"completed_at":"2026-09-13T13:06:54.701Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2292gitlab # [ 439.483426] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cab9e5f8459b034c92cb60c","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cab9e5f8459b034c92cb60c: start","job_status":"start","queue_duration_s":2.551798,"scheduling_latency_s":2.551832}2293gitlab # [ 439.495887] gitaly[919]: time="2026-09-13T13:06:56.172Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P diskcache=db1d9f70-f552-4d6d-9eb1-88fb32db4745 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:26.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.022" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2294gitlab # [ 439.505634] gitaly[919]: time="2026-09-13T13:06:56.172Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=127 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=6889 command.oublock=0 command.real_time_ms=136 command.spawn_token_fork_ms=1 command.system_time_ms=73 command.user_time_ms=54 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:26.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.022" grpc.time_ms=149.872 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2295gitlab # [ 439.518888] nixbot[3565]: {"ts": "2026-09-13T13:06:56.376Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 \"HTTP/1.1 201 Created\""}2296gitlab # [ 439.521752] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cab9e5f8459b034c92cb60c","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cab9e5f8459b034c92cb60c: done: 0.023802 sec","job_status":"done","queue_duration_s":2.551798,"scheduling_latency_s":2.551832,"redis_calls":13,"redis_duration_s":0.003432,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023802,"completed_at":"2026-09-13T13:06:54.727Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2297gitlab # [ 439.561962] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e880663a51a36e48c1b61c0a","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e880663a51a36e48c1b61c0a: start","job_status":"start","queue_duration_s":2.5775,"scheduling_latency_s":2.577528}2298gitlab # [ 439.574433] gitlab-workhorse[819]: gitlab 2001:db8:1::1 - - [2026/09/13:13:06:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/d7f612ca5106e30c4e4733fb3b372a4004ee28e2 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1992299gitlab # [ 439.576895] gitaly[919]: time="2026-09-13T13:06:56.199Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:26.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.022" pid=919 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2300gitlab # [ 439.590001] gitaly[919]: time="2026-09-13T13:06:56.199Z" level=info msg="optimized repository" clean_stale_data_time=34.40182ms clean_work_trees_time="158.02µs" commit_graph_time=720ns correlation_id=01M2DE1FZKHZ74G45DY0338X2P gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.94µs" pid=919 prune_time=600ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="1.76µs" storage_name=default system=housekeeping total_time=34.6067ms2301gitlab # [ 439.596075] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e880663a51a36e48c1b61c0a","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e880663a51a36e48c1b61c0a: done: 0.019749 sec","job_status":"done","queue_duration_s":2.5775,"scheduling_latency_s":2.577528,"redis_calls":13,"redis_duration_s":0.003004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00194,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019749,"completed_at":"2026-09-13T13:06:54.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2302gitlab # [ 439.634662] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ca52806285284a70a9e7670","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ca52806285284a70a9e7670: start","job_status":"start","queue_duration_s":2.598981,"scheduling_latency_s":2.599009}2303gitlab # [ 439.647083] gitaly[919]: time="2026-09-13T13:06:56.199Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T13:07:26.021" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.022" pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2304gitlab # [ 439.656403] gitaly[919]: time="2026-09-13T13:06:56.260Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1GHGRQWCP59BT5TS043C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:26.258" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:56.259" grpc.time_ms=0.964 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2305gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses2306gitlab # [ 439.668775] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ca52806285284a70a9e7670","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ca52806285284a70a9e7670: done: 0.018695 sec","job_status":"done","queue_duration_s":2.598981,"scheduling_latency_s":2.599009,"redis_calls":13,"redis_duration_s":0.00345,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002429,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018695,"completed_at":"2026-09-13T13:06:54.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2307gitlab # [ 439.706921] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4250e0f4007a6369d5a3fe","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4250e0f4007a6369d5a3fe: start","job_status":"start","queue_duration_s":2.619405,"scheduling_latency_s":2.619432}2308gitlab # [ 439.718718] gitaly[919]: time="2026-09-13T13:06:56.275Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T13:07:26.274" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:56.274" grpc.time_ms=1.055 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2309gitlab # [ 439.730455] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4250e0f4007a6369d5a3fe","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4250e0f4007a6369d5a3fe: done: 0.017083 sec","job_status":"done","queue_duration_s":2.619405,"scheduling_latency_s":2.619432,"redis_calls":13,"redis_duration_s":0.002855,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000969,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010553,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017083,"completed_at":"2026-09-13T13:06:54.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2310gitlab # [ 439.770198] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.790Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9378c1e91fdb1b499965233","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9378c1e91fdb1b499965233: start","job_status":"start","queue_duration_s":2.63834,"scheduling_latency_s":2.63836}2311gitlab # [ 439.782824] gitaly[919]: time="2026-09-13T13:06:56.318Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=10 component=gitaly.StreamServerInterceptor correlation_id=01M2DE1GHGRQWCP59BT5TS043C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T13:07:06.244" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T13:06:56.304" grpc.time_ms=14.007 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2312gitlab # [ 439.796438] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9378c1e91fdb1b499965233","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9378c1e91fdb1b499965233: done: 0.016927 sec","job_status":"done","queue_duration_s":2.63834,"scheduling_latency_s":2.63836,"redis_calls":13,"redis_duration_s":0.0030229999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001027,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016927,"completed_at":"2026-09-13T13:06:54.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2313gitlab # [ 439.836154] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.808Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5bddff3ff8f14b1b04c6de4","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5bddff3ff8f14b1b04c6de4: start","job_status":"start","queue_duration_s":2.656709,"scheduling_latency_s":2.656727}2314gitlab # [ 439.848377] gitaly[919]: time="2026-09-13T13:06:56.319Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2DE1FZKHZ74G45DY0338X2P grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T13:07:26.317" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T13:06:56.318" grpc.time_ms=0.869 pid=919 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2315gitlab # [ 439.860693] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.824Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5bddff3ff8f14b1b04c6de4","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5bddff3ff8f14b1b04c6de4: done: 0.015691 sec","job_status":"done","queue_duration_s":2.656709,"scheduling_latency_s":2.656727,"redis_calls":13,"redis_duration_s":0.002611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000892,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001719,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015691,"completed_at":"2026-09-13T13:06:54.824Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2316gitlab # [ 439.900656] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6967ea9ec4b125b97faa260","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6967ea9ec4b125b97faa260: start","job_status":"start","queue_duration_s":2.673947,"scheduling_latency_s":2.673965}2317gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/d7f612ca5106e30c4e4733fb3b372a4004ee28e2/statuses, in 0.23 seconds)2318(finished: subtest: eval and build post commit statuses, in 11.35 seconds)2319(finished: run the VM test script, in 510.73 seconds)2320gitlab # [ 439.913169] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.841Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6967ea9ec4b125b97faa260","created_at":"2026-09-13T13:06:52.002Z","trace_propagation_headers":{"sentry-trace":"c85491e676604b6b8b1a83379cc2f573-ab37f0e614a848bb","baggage":"sentry-trace_id=c85491e676604b6b8b1a83379cc2f573,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"7d86665da3644426382c6f63bd76bd4c","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T13:06:52.152Z","job_size_bytes":2,"pid":3435,"sidekiq_tid":"m8lz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6967ea9ec4b125b97faa260: done: 0.015537 sec","job_status":"done","queue_duration_s":2.673947,"scheduling_latency_s":2.673965,"redis_calls":13,"redis_duration_s":0.002871,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009606,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015537,"completed_at":"2026-09-13T13:06:54.841Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321test script finished in 510.76s2322cleanup2323kill QemuMachine (pid 46)2324gitlab # [ 439.958539] sidekiq-cluster[3435]: {"severity":"INFO","time":"2026-09-13T13:06:54.843Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4dd37f6fb640e6f7436edbc9","created_at":"2026-09-13T13:06:52.681Z","trace_propagation_headers":{"seqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2325(finished: cleanup, in 1.59 seconds)2326additionally exposed symbols:2327 gitlab,2328 vlan1,2329 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2330[]2331[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-13T13:05:19.900Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2332[]2333[]2334[]2335[]2336[]2337[]2338[]2339[]2340[]2341[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-13T13:06:45.445122Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2342[]2343[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2344[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2345[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2346[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2347[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2348[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2349[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2350[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2351[{'id': 5, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-13T13:06:45.849Z', 'started_at': None, 'finished_at': '2026-09-13T13:06:55.817Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'd7f612ca5106e30c4e4733fb3b372a4004ee28e2', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-13T13:06:56.107Z', 'started_at': None, 'finished_at': '2026-09-13T13:06:56.346Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]