vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #352
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.mhx3VjDx8R', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: a7c89c49-f72d-4c09-8ef7-8a199500bde916(finished: start all VMs, in 72.66 seconds)17gitlab # Superblock backups stored on blocks:18gitlab: waiting for unit setup-gitlab.service19gitlab # 32768, 98304, 163840, 22937620gitlab: waiting for the VM to finish booting21gitlab # 22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/78hxbdjzk789wa5gdm889n9h5pc3fjcq-nixos-system-gitlab-test/init regInfo=/nix/store/0h1xhxci47i57c7k7axhf1hcjl7imzij-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0h1xhxci47i57c7k7axhf1hcjl7imzij-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f40000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f50000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000028] arm-pv: using stolen time PV108gitlab # [ 0.000358] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000548] Console: colour dummy device 80x25110gitlab # [ 0.000555] printk: legacy console [tty0] enabled111gitlab # [ 0.000741] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000748] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000834] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.000973] landlock: Up and running.115gitlab # [ 0.000975] Yama: becoming mindful.116gitlab # [ 0.001382] LSM support for eBPF active117gitlab # [ 0.001552] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001667] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.002868] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003555] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003559] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.003712] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.004739] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.004833] EFI services will not be available.125gitlab # [ 0.004996] smp: Bringing up secondary CPUs ...126gitlab # [ 0.005639] Detected PIPT I-cache on CPU1127gitlab # [ 0.005740] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.005872] GICv3: CPU1: using allocated LPI pending table @0x0000000100f60000129gitlab # [ 0.006004] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007130] Detected PIPT I-cache on CPU2131gitlab # [ 0.007239] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007370] GICv3: CPU2: using allocated LPI pending table @0x0000000100f70000133gitlab # [ 0.007500] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008575] Detected PIPT I-cache on CPU3135gitlab # [ 0.008693] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008825] GICv3: CPU3: using allocated LPI pending table @0x0000000100f80000137gitlab # [ 0.008954] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.009483] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.009510] SMP: Total of 4 processors activated.140gitlab # [ 0.009512] CPU: All CPU(s) started at EL1141gitlab # [ 0.009522] CPU features: detected: Branch Target Identification142gitlab # [ 0.009526] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.009529] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.009532] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.009536] CPU features: detected: Common not Private translations146gitlab # [ 0.009539] CPU features: detected: CRC32 instructions147gitlab # [ 0.009542] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.009545] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.009548] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.009551] CPU features: detected: E0PD151gitlab # [ 0.009553] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.009556] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.009560] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.009563] CPU features: detected: Fine Grained Traps155gitlab # [ 0.009566] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.009571] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.009573] CPU features: detected: LSE atomic instructions158gitlab # [ 0.009576] CPU features: detected: Privileged Access Never159gitlab # [ 0.009579] CPU features: detected: PMUv3160gitlab # [ 0.009581] CPU features: detected: RAS Extension Support161gitlab # [ 0.009584] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.009587] CPU features: detected: Random Number Generator163gitlab # [ 0.009589] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.009592] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.009595] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.009599] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.009728] alternatives: applying system-wide alternatives168gitlab # [ 0.012640] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.012847] Memory: 5963764K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.014342] devtmpfs: initialized171gitlab # [ 0.018357] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.018430] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.018609] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.018614] 0 pages in range for non-PLT usage175gitlab # [ 0.018615] 508288 pages in range for PLT usage176gitlab # [ 0.018767] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.019577] DMI not present or invalid.178gitlab # [ 0.022724] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.025326] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.025610] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.027449] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.027471] audit: initializing netlink subsys (disabled)183gitlab # [ 0.027774] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.028205] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.028208] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.028212] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.028215] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.028218] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.028300] cpuidle: using governor ladder190gitlab # [ 0.028361] cpuidle: using governor menu191gitlab # [ 0.028562] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.028775] ASID allocator initialised with 65536 entries193gitlab # [ 0.029854] Serial: AMBA PL011 UART driver194gitlab # [ 0.035025] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.035328] printk: console [ttyAMA0] enabled196gitlab # [ 0.047471] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.047489] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.047499] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.047507] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.047516] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.047523] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.047532] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.047540] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.058867] fbcon: Taking over console205gitlab # [ 0.058878] ACPI: Interpreter disabled.206gitlab # [ 0.059900] iommu: Default domain type: Translated207gitlab # [ 0.059916] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.069624] SCSI subsystem initialized209gitlab # [ 0.070010] usbcore: registered new interface driver usbfs210gitlab # [ 0.070032] usbcore: registered new interface driver hub211gitlab # [ 0.070048] usbcore: registered new device driver usb212gitlab # [ 0.070299] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.070303] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.070320] PTP clock support registered215gitlab # [ 0.070357] EDAC MC: Ver: 3.0.0216gitlab # [ 0.070874] scmi_core: SCMI protocol bus registered217gitlab # [ 0.071891] FPGA manager framework218gitlab # [ 0.072654] vgaarb: loaded219gitlab # [ 0.074718] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.075132] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.075155] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.075515] netfs: FS-Cache loaded223gitlab # [ 0.075614] pnp: PnP ACPI: disabled224gitlab # [ 0.078694] NET: Registered PF_INET protocol family225gitlab # [ 0.079593] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.108404] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.108451] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.108499] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.108815] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.109365] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.109454] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.109503] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.109613] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.109797] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.109814] NET: Registered PF_XDP protocol family236gitlab # [ 0.109827] PCI: CLS 0 bytes, default 64237gitlab # [ 0.110090] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.122868] kvm [1]: HYP mode not available239gitlab # [ 0.210032] Initialise system trusted keyrings240gitlab # [ 0.210382] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.211489] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.211560] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.230141] Key type asymmetric registered244gitlab # [ 0.230147] Asymmetric key parser 'x509' registered245gitlab # [ 0.230288] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.230604] io scheduler mq-deadline registered247gitlab # [ 0.230613] io scheduler kyber registered248gitlab # [ 0.237580] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.238953] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.239574] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.239593] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.239608] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.239617] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.239643] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.239682] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.239835] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.239846] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.239852] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.239857] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.239862] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.239952] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.240850] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.241278] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.241302] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.241362] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.241386] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.242498] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.242939] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.242962] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.243014] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.244053] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.244486] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.244509] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.244560] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.245560] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.245970] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.245991] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.246047] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.247095] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.247504] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.247526] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.247581] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.248535] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.248918] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.248940] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.248994] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.250017] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.250421] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.250442] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.250491] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.250519] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.251558] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.252008] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.252064] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.253039] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.253436] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.253491] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.254308] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.254744] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.255218] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.255696] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.255719] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.255773] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.256762] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.257171] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.257192] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.257241] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.258392] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.258403] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.258409] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.258495] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.258588] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.258675] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.258768] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.258862] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.258945] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.259036] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.259126] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.259215] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.259336] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.259414] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.259444] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.259475] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.259507] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.259545] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.259576] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.259607] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.259637] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.259667] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.259699] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.259731] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.259769] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.259800] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.259830] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.259861] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.259890] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.259920] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.259952] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.259990] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.260021] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.260052] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.260090] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.260099] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.260104] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.262369] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.280745] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.284078] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.287110] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.290300] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.296617] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.299597] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.302503] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.305624] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.312084] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.314768] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.318174] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.326379] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.328898] msm_serial: driver initialized359gitlab # [ 0.329142] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.329213] STM32 USART driver initialized361gitlab # [ 0.356714] loop: module loaded362gitlab # [ 0.357005] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.359556] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.361981] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.364318] virtio_blk virtio5: [vdb] 12284704 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.367044] megasas: 07.734.00.00-rc1367gitlab # [ 0.367931] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.370308] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.370334] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.372468] Using buffer write method371gitlab # [ 0.372515] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.374937] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.374963] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.377183] Using buffer write method375gitlab # [ 0.377209] Concatenating MTD devices:376gitlab # [ 0.377213] (0): "0.flash"377gitlab # [ 0.377216] (1): "0.flash"378gitlab # [ 0.377218] into device "0.flash"379gitlab # [ 0.528258] Freeing initrd memory: 27184K380gitlab # [ 0.542573] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.549610] thunder_xcv, ver 1.0382gitlab # [ 0.549699] thunder_bgx, ver 1.0383gitlab # [ 0.549748] nicpf, ver 1.0384gitlab # [ 0.550979] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.550988] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.551043] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.551054] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.551103] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.551109] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.551156] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.551164] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.551432] sky2: driver version 1.30393gitlab # [ 0.554897] usbcore: registered new interface driver usb-storage394gitlab # [ 0.555116] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.555150] usbserial: USB Serial support registered for generic396gitlab # [ 0.555853] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.555927] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.556305] hv_vmbus: registering driver hyperv_keyboard399gitlab # [ 0.556559] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000400gitlab # [ 0.558191] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.558237] rtc-pl031 9010000.pl031: setting system clock to 2026-09-06T16:30:48 UTC (1788712248)402gitlab # [ 0.558897] i2c_dev: i2c /dev entries driver403gitlab # [ 0.564291] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.564302] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.564806] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.565523] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.566856] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00408gitlab # [ 0.567561] hub 1-0:1.0: USB hub found409gitlab # [ 0.567589] hub 1-0:1.0: 6 ports detected410gitlab # [ 0.568659] hid: raw HID events driver (C) Jiri Kosina411gitlab # [ 0.569194] usbcore: registered new interface driver usbhid412gitlab # [ 0.569202] usbhid: USB HID core driver413gitlab # [ 0.571737] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.574421] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.574808] NET: Registered PF_INET6 protocol family416gitlab # [ 0.576154] Segment Routing with IPv6417gitlab # [ 0.576178] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.576292] NET: Registered PF_PACKET protocol family419gitlab # [ 0.576675] 9pnet: Installing 9P2000 support420gitlab # [ 0.579468] Key type dns_resolver registered421gitlab # [ 0.589279] registered taskstats version 1422gitlab # [ 0.589571] Loading compiled-in X.509 certificates423gitlab # [ 0.601586] Demotion targets for Node 0: null424gitlab # [ 0.601754] Key type .fscrypt registered425gitlab # [ 0.601759] Key type fscrypt-provisioning registered426gitlab # [ 0.601944] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.601965] ima: Allocated hash algorithm: sha1428gitlab # [ 0.601992] ima: No architecture policies found429gitlab # [ 0.603040] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.637446] clk: Disabling unused clocks431gitlab # [ 0.637466] PM: genpd: Disabling unused power domains432gitlab # [ 0.641194] Freeing unused kernel memory: 4736K433gitlab # [ 0.641423] Run /init as init process434gitlab # [ 0.673089] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.814817] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 0.967594] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.007539] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.007611] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.007695] systemd[1]: Detected architecture arm64.440gitlab # [ 1.007713] systemd[1]: Running in initrd.441gitlab # [ 1.008593] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.008856] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.099480] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.218809] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.280678] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.377514] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.379044] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2448gitlab # [ 1.379508] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0449gitlab # [ 1.404401] systemd[1]: Created slice Slice /system/modprobe.450gitlab # [ 1.404863] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.451gitlab # [ 1.404929] systemd[1]: Expecting device /dev/disk/by-label/nix-store...452gitlab # [ 1.404955] systemd[1]: Expecting device /dev/disk/by-label/nixos...453gitlab # [ 1.405000] systemd[1]: Reached target Path Units.454gitlab # [ 1.405037] systemd[1]: Reached target Slice Units.455gitlab # [ 1.405072] systemd[1]: Reached target Swaps.456gitlab # [ 1.405105] systemd[1]: Reached target Timer Units.457gitlab # [ 1.405420] systemd[1]: Listening on D-Bus System Message Bus Socket.458gitlab # [ 1.405726] systemd[1]: Listening on Journal Socket (/dev/log).459gitlab # [ 1.406023] systemd[1]: Listening on Journal Sockets.460gitlab # [ 1.406304] systemd[1]: Listening on udev Control Socket.461gitlab # [ 1.406543] systemd[1]: Listening on udev Kernel Socket.462gitlab # [ 1.406584] systemd[1]: Reached target Socket Units.463gitlab # [ 1.409776] systemd[1]: Starting Create List of Static Device Nodes...464gitlab # [ 1.413100] systemd[1]: Starting Load Kernel Module 9pnet_virtio...465gitlab # [ 1.413234] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs466gitlab # [ 1.423086] systemd[1]: Mounting Kernel Configuration File System...467gitlab # [ 1.447709] systemd[1]: Starting Journal Service...468gitlab # [ 1.450572] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.450822] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.459072] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.462291] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.466065] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.471098] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.472524] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.487081] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...476gitlab # [ 1.499712] systemd-journald[96]: Collecting audit messages is disabled.477gitlab # [ 1.511406] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0478gitlab # [ 1.511688] [drm] features: -virgl +edid -resource_blob -host_visible479gitlab # [ 1.511693] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.480gitlab # [ 1.511719] [drm] features: -context_init481gitlab # [ 1.512847] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.482gitlab # [ 1.513220] [drm] number of scanouts: 1483gitlab # [ 1.513259] [drm] number of cap sets: 0484gitlab # [ 1.518782] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev485gitlab # [ 1.519035] systemd[1]: Starting Create Static Device Nodes in /dev...486gitlab # [ 1.531049] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic487gitlab # [ 1.531077] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0488gitlab # [ 1.532092] systemd[1]: Finished Create Static Device Nodes in /dev.489gitlab # [ 1.532321] systemd[1]: Reached target Preparation for Local File Systems.490gitlab # [ 1.532356] systemd[1]: Reached target Local File Systems.491gitlab # [ 1.538995] systemd[1]: Starting Rule-based Manager for Device Events and Files...492gitlab # [ 1.545801] Console: switching to colour frame buffer device 160x50493gitlab # [ 1.552422] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device494gitlab # [ 1.570703] systemd[1]: Started Journal Service.495gitlab # [ 1.570596] systemd-modules-load[97]: Using 4 probe threads496gitlab # [ 1.573895] systemd-modules-load[97]: Module 'virtio_balloon' is built in497gitlab # [ 1.575294] systemd-modules-load[97]: Module 'virtio_console' is built in498gitlab # [ 1.576830] systemd-modules-load[97]: Module 'virtio_rng' is built in499gitlab # [ 1.580292] systemd-modules-load[97]: Inserted module 'dm_mod'500gitlab # [ 1.581588] systemd-modules-load[97]: Inserted module 'virtio_gpu'501gitlab # [ 1.582787] systemd[1]: Finished Load Kernel Modules.502gitlab # [ 1.583883] systemd[1]: Starting Apply Kernel Variables...503gitlab # [ 1.585289] systemd[1]: Starting Create System Files and Directories...504gitlab # [ 1.596465] systemd-udevd[106]: Using default interface naming scheme 'v261'.505gitlab # [ 1.607650] systemd[1]: Finished Create System Files and Directories.506gitlab # [ 1.613656] systemd[1]: Finished Apply Kernel Variables.507gitlab # [ 1.616388] systemd[1]: Started Rule-based Manager for Device Events and Files.508gitlab # [ 1.632712] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.681647] systemd-vconsole-setup[132]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.686731] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.087469] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.090179] systemd[1]: Reached target System Initialization.513gitlab # [ 2.092779] systemd[1]: Reached target Basic System.514gitlab # [ 2.303023] (udev-worker)[123]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.515gitlab # [ 2.307267] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line.516gitlab # [ 2.341478] (udev-worker)[129]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.348219] systemd[1]: Found device /dev/disk/by-label/nixos.518gitlab # [ 2.356556] systemd[1]: Found device /dev/disk/by-label/nix-store.519gitlab # [ 2.358310] systemd[1]: Reached target Initrd Root Device.520gitlab # [ 2.360366] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...521gitlab # [ 2.413995] systemd-fsck[154]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.421536] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.459405] systemd[1]: Mounting /sysroot...524gitlab # [ 2.510250] EXT4-fs (vda): mounted filesystem a7c89c49-f72d-4c09-8ef7-8a199500bde9 r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.512575] systemd[1]: Mounted /sysroot.526gitlab # [ 2.513553] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.516738] systemd[1]: Mounting /sysroot/nix/.ro-store...528gitlab # [ 2.519706] systemd[1]: Mounting /sysroot/nix/.rw-store...529gitlab # [ 2.526083] systemd[1]: Mounting /sysroot/run...530gitlab # [ 2.552133] systemd[1]: Mounting /sysroot/tmp/shared...531gitlab # [ 2.561659] systemd[1]: Mounting /sysroot/tmp/xchg...532gitlab # [ 2.563503] systemd[1]: Starting Mountpoints Configured in the Real Root...[ 2.569479] erofs (device vdb): mounted with root inode @ nid 128.533gitlab # 534gitlab # [ 2.571414] systemd[1]: Mounted /sysroot/nix/.rw-store.535gitlab # [ 2.573311] systemd[1]: Mounted /sysroot/run.536gitlab # [ 2.575607] systemd[1]: Mounted /sysroot/nix/.ro-store.537gitlab # [ 2.580121] systemd[1]: Mounted /sysroot/tmp/shared.538gitlab # [ 2.581239] systemd-sysroot-fstab-check[168]: /sysroot should be mounted in the initrd, will request daemon-reload.539gitlab # [ 2.584376] systemd[1]: Starting rw-sysroot-nix-store.service...540gitlab # [ 2.588957] systemd[1]: Mounted /sysroot/tmp/xchg.541gitlab # [ 2.590743] systemd[1]: Reload requested from client PID 168 ('systemd-sysroot') (unit initrd-parse-etc.service)...542gitlab # [ 2.594346] systemd[1]: Reloading...543gitlab # [ 2.727844] systemd[1]: Reloading finished in 136 ms.544gitlab # [ 2.782010] systemd-sysroot-fstab-check[168]: Requesting initrd-fs.target/start/replace...545gitlab # [ 2.783384] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.546gitlab # [ 2.785191] systemd[1]: Finished rw-sysroot-nix-store.service.547gitlab # [ 2.786311] systemd-sysroot-fstab-check[168]: Requesting swap.target/start/replace...548gitlab # [ 2.789939] systemd[1]: Starting rw-sysroot-nix-store.service...549gitlab # [ 2.790953] systemd[1]: initrd-parse-etc.service: Deactivated successfully.550gitlab # [ 2.792810] systemd[1]: Finished Mountpoints Configured in the Real Root.551gitlab # [ 2.793825] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.552gitlab # [ 2.815072] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.553gitlab # [ 2.817006] systemd[1]: Finished rw-sysroot-nix-store.service.554gitlab # [ 3.057846] (udev-worker)[141]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory555gitlab # [ 3.061395] (udev-worker)[141]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory556gitlab # [ 3.077177] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.557gitlab # [ 3.078437] systemd[1]: Stopped Virtual Console Setup.558gitlab # [ 3.079228] systemd[1]: Stopping Virtual Console Setup...559gitlab # [ 3.080088] systemd[1]: Starting Virtual Console Setup...560gitlab # [ 3.121703] systemd-vconsole-setup[197]: Configuration of first virtual console was skipped, ignoring remaining ones.561gitlab # [ 3.124367] systemd[1]: Finished Virtual Console Setup.562gitlab # [ 3.470559] systemd[1]: Mounting /sysroot/nix/store...563gitlab # [ 3.590083] systemd[1]: Mounted /sysroot/nix/store.564gitlab # [ 3.592679] systemd[1]: Reached target Initrd File Systems.565gitlab # [ 3.596606] systemd[1]: Starting Find NixOS closure...566gitlab # [ 3.601644] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...567gitlab # [ 3.677181] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.568gitlab # [ 3.685289] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.569gitlab # [ 3.718554] systemd[1]: Finished Find NixOS closure.570gitlab # [ 3.721570] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 3.727819] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 3.818247] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 3.821131] systemd[1]: Stopped target Basic System.574gitlab # [ 3.823416] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 3.826030] systemd[1]: Stopped target Path Units.576gitlab # [ 3.828309] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 3.831627] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 3.835103] systemd[1]: Stopped target Slice Units.579gitlab # [ 3.837511] systemd[1]: Stopped target Socket Units.580gitlab # [ 3.839784] systemd[1]: Stopped target System Initialization.581gitlab # [ 3.842500] systemd[1]: Stopped target Swaps.582gitlab # [ 3.844606] systemd[1]: Stopped target Timer Units.583gitlab # [ 3.850877] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 3.853780] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 3.859353] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 3.865588] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 3.869401] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 3.872425] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 3.875550] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 3.878215] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 3.880068] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 3.882682] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 3.884749] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 3.887413] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 3.890235] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 3.892866] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 3.895015] systemd[1]: Stopped target Local File Systems.598gitlab # [ 3.897003] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 3.899263] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 3.901854] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 3.903823] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 3.906303] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 3.908733] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 3.910284] systemd[1]: initrd-cleanup.service: Deactivated successfully.605gitlab # [ 3.912248] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.606gitlab # [ 3.914030] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.607gitlab # [ 3.915998] systemd[1]: Finished rw-sysroot-nix-store.service.608gitlab # [ 3.917590] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.609gitlab # [ 3.919434] systemd[1]: Finished Load Kernel Module 9pnet_virtio.610gitlab # [ 3.921024] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 3.922621] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 3.924412] systemd[1]: systemd-udevd.service: Consumed 2.786s CPU time over 2.370s wall clock time, 31.2M memory peak.613gitlab # [ 3.926859] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 3.928566] systemd[1]: Closed udev Control Socket.615gitlab # [ 3.930261] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 3.931633] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 3.933507] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 3.934901] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 3.936740] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 3.938343] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 3.939889] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 4.004987] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 4.009167] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 4.011948] systemd[1]: Reached target Switch Root.625gitlab # [ 4.014334] systemd[1]: Starting NixOS Activation...626gitlab # [ 4.282477] initrd-nixos-activation-start[224]: booting system configuration /nix/store/78hxbdjzk789wa5gdm889n9h5pc3fjcq-nixos-system-gitlab-test627gitlab # [ 4.373833] initrd-nixos-activation-start[224]: running activation script...628gitlab # [ 4.861406] initrd-nixos-activation-start[247]: setting up /etc...629gitlab # [ 5.125208] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.630gitlab # [ 5.128918] systemd[1]: Finished NixOS Activation.631gitlab # [ 5.135793] systemd[1]: Starting Switch Root...632gitlab # [ 5.230730] systemd[1]: Switching root.633gitlab # [ 5.423627] systemd-journald[96]: Received SIGTERM from PID 1 (systemd).634gitlab # [ 6.158685] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)635gitlab # [ 6.158936] systemd[1]: Detected virtualization qemu.636gitlab # [ 6.159050] systemd[1]: Detected architecture arm64.637gitlab # [ 6.159246] systemd[1]: Detected first boot.638gitlab # [ 6.160767] systemd[1]: Initializing machine ID from random generator.639gitlab # [ 6.438148] systemd[1]: bpf-restrict-fs: LSM BPF program attached640gitlab # [ 6.574439] systemd[1]: Applying preset policy.641gitlab # [ 6.815626] systemd[1]: Populated /etc with preset unit settings.642gitlab # [ 7.043228] systemd[1]: initrd-switch-root.service: Deactivated successfully.643gitlab # [ 7.043936] systemd[1]: Stopped initrd-switch-root.service.644gitlab # [ 7.046972] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.645gitlab # [ 7.051318] systemd[1]: Created slice Slice /system/getty.646gitlab # [ 7.058655] systemd[1]: Created slice GitLab DevOps Platform Slice.647gitlab # [ 7.061882] systemd[1]: Created slice User and Session Slice.648gitlab # [ 7.062276] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.649gitlab # [ 7.062545] systemd[1]: Started Forward Password Requests to Wall Directory Watch.650gitlab # [ 7.062660] systemd[1]: Expecting device /dev/hvc0...651gitlab # [ 7.062706] systemd[1]: Expecting device /dev/ttyAMA0...652gitlab # [ 7.066523] systemd[1]: Reached target Local Encrypted Volumes.653gitlab # [ 7.067781] systemd[1]: Stopped target initrd-fs.target.654gitlab # [ 7.068919] systemd[1]: Stopped target initrd-root-fs.target.655gitlab # [ 7.070063] systemd[1]: Stopped target initrd-switch-root.target.656gitlab # [ 7.071228] systemd[1]: Reached target Virtual Machines and Containers.657gitlab # [ 7.072378] systemd[1]: Reached target Path Units.658gitlab # [ 7.073507] systemd[1]: Reached target Remote File Systems.659gitlab # [ 7.074691] systemd[1]: Reached target Slice Units.660gitlab # [ 7.076694] systemd[1]: Reached target Swaps.661gitlab # [ 7.080873] systemd[1]: Listening on Query the User Interactively for a Password.662gitlab # [ 7.085305] systemd[1]: Listening on Process Core Dump Socket.663gitlab # [ 7.088545] systemd[1]: Listening on Credential Encryption/Decryption.664gitlab # [ 7.092369] systemd[1]: Listening on Factory Reset Management.665gitlab # [ 7.093812] systemd[1]: Listening on Hostname Service Socket.666gitlab # [ 7.103798] systemd[1]: Starting Journal Log Access Socket...667gitlab # [ 7.106069] systemd[1]: Listening on Journal Audit Socket.668gitlab # [ 7.110274] systemd[1]: Listening on Console Output Muting Service Socket.669gitlab # [ 7.112126] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.670gitlab # [ 7.113350] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os671gitlab # [ 7.114369] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki672gitlab # [ 7.120451] systemd[1]: Listening on Disk Repartitioning Service Socket.673gitlab # [ 7.121873] systemd[1]: Listening on udev Control Socket.674gitlab # [ 7.123328] systemd[1]: Listening on udev Varlink Socket.675gitlab # [ 7.163565] systemd[1]: Mounting Huge Pages File System...676gitlab # [ 7.169465] systemd[1]: Mounting POSIX Message Queue File System...677gitlab # [ 7.177116] systemd[1]: Mounting Kernel Debug File System...678gitlab # [ 7.186996] systemd[1]: Mounting Kernel Trace File System...679gitlab # [ 7.194482] systemd[1]: Starting Create List of Static Device Nodes...680gitlab # [ 7.202415] systemd[1]: Starting Load Kernel Module 9pnet_virtio...681gitlab # [ 7.207127] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs682gitlab # [ 7.227176] systemd[1]: Mounting Kernel Configuration File System...683gitlab # [ 7.228280] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm684gitlab # [ 7.229159] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore685gitlab # [ 7.251389] systemd[1]: Starting Load Kernel Module fuse...686gitlab # [ 7.252671] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67687gitlab # [ 7.264459] systemd[1]: Starting Journal Service...688gitlab # [ 7.270474] systemd[1]: Starting Load Kernel Modules...689gitlab # [ 7.279501] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...690gitlab # [ 7.287078] systemd[1]: Starting Remount Root and Kernel File Systems...691gitlab # [ 7.288286] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os692gitlab # [ 7.299368] systemd[1]: Starting Coldplug All udev Devices...693gitlab # [ 7.311762] systemd[1]: Listening on Journal Log Access Socket.694gitlab # [ 7.312999] systemd-journald[318]: Collecting audit messages is enabled.695gitlab # [ 7.313016] fuse: init (API version 7.45)696gitlab # [ 7.313165] systemd[1]: Mounted Huge Pages File System.697gitlab # [ 7.313559] systemd[1]: Mounted POSIX Message Queue File System.698gitlab # [ 7.315382] systemd[1]: Mounted Kernel Debug File System.699gitlab # [ 7.316393] systemd[1]: Mounted Kernel Trace File System.700gitlab # [ 7.331255] systemd[1]: Finished Create List of Static Device Nodes.701gitlab # [ 7.333102] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.702gitlab # [ 7.331734] systemd[1]: Queued start job for default target Multi-User System.703gitlab # [ 7.339974] systemd[1]: Finished Load Kernel Module 9pnet_virtio.704gitlab # [ 7.337680] systemd[1]: Unnecessary job was removed for /dev/vdb.[ 7.343149] systemd[1]: Started Journal Service.705gitlab # 706gitlab # [ 7.346545] systemd[1]: systemd-journald.service: Deactivated successfully.707gitlab # [ 7.350505] systemd-modules-load[319]: Using 3 probe threads708gitlab # [ 7.355259] systemd-modules-load[319]: Module 'atkbd' is built in709gitlab # [ 7.357892] systemd-modules-load[319]: Module 'loop' is built in710gitlab # [ 7.359979] systemd[1]: Mounted Kernel Configuration File System.711gitlab # [ 7.362168] systemd[1]: modprobe@fuse.service: Deactivated successfully.712gitlab # [ 7.364381] systemd[1]: Finished Load Kernel Module fuse.713gitlab # [ 7.366128] systemd-modules-load[319]: Inserted module 'tls'714gitlab # [ 7.367884] systemd[1]: Mounting FUSE Control File System...715gitlab # [ 7.369613] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...716gitlab # [ 7.371814] systemd[1]: Finished Load Kernel Modules.717gitlab # [ 7.376647] EXT4-fs (vda): re-mounted a7c89c49-f72d-4c09-8ef7-8a199500bde9.718gitlab # [ 7.396688] systemd[1]: Starting Firewall...719gitlab # [ 7.400356] systemd[1]: Starting Apply Kernel Variables...720gitlab # [ 7.403507] systemd[1]: Finished Remount Root and Kernel File Systems.721gitlab # [ 7.407206] systemd[1]: Mounted FUSE Control File System.722gitlab # [ 7.408991] systemd-oomd[321]: No swap; memory pressure usage will be degraded723gitlab # [ 7.411398] systemd[1]: Listening on Disk Image Download Service Socket.724gitlab # [ 7.416452] systemd[1]: Starting Flush Journal to Persistent Storage...725gitlab # [ 7.420914] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore726gitlab # [ 7.432096] systemd[1]: Starting Load/Save OS Random Seed...727gitlab # [ 7.433379] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os728gitlab # [ 7.439091] systemd-journald[318]: Received client request to flush runtime journal.729gitlab # [ 7.468327] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.730gitlab # [ 7.472467] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.731gitlab # [ 7.473997] systemd[1]: Starting Create Static Device Nodes in /dev...732gitlab # [ 7.476260] systemd[1]: Finished Apply Kernel Variables.733gitlab # [ 7.477467] systemd[1]: Finished Load/Save OS Random Seed.734gitlab # [ 7.478380] systemd[1]: Reached target First Boot Complete.735gitlab # [ 7.479265] systemd[1]: Finished Flush Journal to Persistent Storage.736gitlab # [ 7.485020] systemd[1]: Finished Create Static Device Nodes in /dev.737gitlab # [ 7.486429] systemd[1]: Reached target Preparation for Local File Systems.738gitlab # [ 7.489604] systemd[1]: Starting Rule-based Manager for Device Events and Files...739gitlab # [ 7.568481] systemd-udevd[363]: Using default interface naming scheme 'v261'.740gitlab # [ 7.612843] systemd[1]: Started Rule-based Manager for Device Events and Files.741gitlab # [ 7.892783] systemd[1]: Finished Firewall.742gitlab # [ 7.999941] systemd[1]: Finished Coldplug All udev Devices.743gitlab # [ 8.051691] systemd[1]: Mounting /run/wrappers...744gitlab # [ 8.063470] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs745gitlab # [ 8.115104] systemd[1]: Mounted /run/wrappers.746gitlab # [ 8.119033] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.747gitlab # [ 8.120418] systemd[1]: Reached target Local File Systems.748gitlab # [ 8.121391] systemd[1]: Listening on Boot Loader Control Service Socket.749gitlab # [ 8.124245] systemd[1]: Starting register-nix-paths.service...750gitlab # [ 8.127052] systemd[1]: Starting Create SUID/SGID Wrappers...751gitlab # [ 8.132232] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.752gitlab # [ 8.145049] systemd[1]: Starting Save Transient machine-id to Disk...753gitlab # [ 8.163472] systemd[1]: Starting Create System Files and Directories...754gitlab # [ 8.191153] (udev-worker)[467]: Network interface NamePolicy= disabled on kernel command line.755gitlab # [ 8.210368] (udev-worker)[416]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.756gitlab # [ 8.214263] (udev-worker)[416]: Network interface NamePolicy= disabled on kernel command line.757gitlab # [ 8.216818] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.758gitlab # [ 8.220343] systemd[1]: Finished Save Transient machine-id to Disk.759gitlab # [ 8.221397] systemd[1]: Finished Create System Files and Directories.760gitlab # [ 8.239515] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.761gitlab # [ 8.248233] systemd[1]: Starting Rebuild Journal Catalog...762gitlab # [ 8.256079] systemd[1]: Starting Record System Boot/Shutdown in UTMP...763gitlab # [ 8.258874] systemd[1]: Condition check resulted in Virtio network device being skipped.764gitlab # [ 8.261225] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore765gitlab # [ 8.263188] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.766gitlab # [ 8.266100] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67767gitlab # [ 8.269481] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore768gitlab # [ 8.272859] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os769gitlab # [ 8.275019] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os770gitlab # [ 8.289108] systemd[1]: Finished Record System Boot/Shutdown in UTMP.771gitlab # [ 8.302947] systemd[1]: Finished Rebuild Journal Catalog.772gitlab # [ 8.308945] systemd[1]: Starting Update is Completed...773gitlab # [ 8.349687] systemd[1]: Finished Update is Completed.774gitlab # [ 8.354466] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.775gitlab # [ 8.360437] systemd[1]: Finished Create SUID/SGID Wrappers.776gitlab # [ 8.361780] systemd[1]: suid-sgid-wrappers.service: Consumed 153ms CPU time over 229ms wall clock time, 6.6M memory peak, 1.9M read from disk.777gitlab # [ 8.371720] mousedev: PS/2 mouse device common for all mice778gitlab # [ 8.624938] systemd[1]: Finished register-nix-paths.service.779gitlab # [ 8.626237] systemd[1]: Reached target System Initialization.780gitlab # [ 8.627480] systemd[1]: Started Discard unused filesystem blocks once a week.781gitlab # [ 8.630067] systemd[1]: Started logrotate.timer.782gitlab # [ 8.630935] systemd[1]: Started Daily Cleanup of Temporary Directories.783gitlab # [ 8.632058] systemd[1]: Reached target Timer Units.784gitlab # [ 8.633346] systemd[1]: Listening on D-Bus System Message Bus Socket.785gitlab # [ 8.634866] systemd[1]: Listening on Nix Daemon Socket.786gitlab # [ 8.637859] systemd[1]: Starting nixbot CI service socket...787gitlab # [ 8.638927] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.788gitlab # [ 8.642534] systemd[1]: Starting D-Bus System Message Bus...789gitlab # [ 8.643610] systemd[1]: Listening on nixbot CI service socket.790gitlab # [ 8.645845] systemd[1]: Reached target Socket Units.791gitlab # [ 8.678498] dbus-broker-launch[560]: Looking up NSS user entry for 'systemd-timesync'...792gitlab # [ 8.680358] dbus-broker-launch[560]: NSS returned no entry for 'systemd-timesync'793gitlab # [ 8.682256] dbus-broker-launch[560]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"794gitlab # [ 8.685536] systemd[1]: Started D-Bus System Message Bus.795gitlab # [ 8.697764] dbus-broker-launch[560]: Ready796gitlab # [ 8.699152] systemd[1]: Reached target Basic System.797gitlab # [ 8.706503] systemd[1]: Started backdoor.service.798gitlab # [ 8.708564] systemd[1]: Starting gitlab-config.service...799gitlab # [ 8.724438] systemd[1]: Starting Import lastlog data into lastlog2 database...800gitlab # [ 8.727641] systemd[1]: Starting Logrotate configuration check...801gitlab # connecting to host...802gitlab # [ 8.748238] systemd[1]: Starting Name Service Cache Daemon (nsncd)...803gitlab # [ 8.750127] logrotate[565]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.804gitlab # [ 8.755605] logrotate[565]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf805gitlab # [ 8.757396] logrotate[565]: note: 'monthly' overrides previously specified 'weekly'806gitlab # [ 8.758730] logrotate[565]: Reading state from file: /var/lib/logrotate.status807gitlab # [ 8.760516] logrotate[565]: state file /var/lib/logrotate.status does not exist808gitlab # [ 8.769394] logrotate[565]: Allocating hash table for state file, size 64 entries809gitlab # [ 8.773517] logrotate[565]: Handling 4 logs810gitlab # [ 8.779895] logrotate[565]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed811gitlab # [ 8.783895] logrotate[565]: considering log /var/log/btmp812gitlab # [ 8.785019] logrotate[565]: Creating new state813gitlab # [ 8.785722] logrotate[565]: Now: 2026-09-06 16:30814gitlab # [ 8.786472] logrotate[565]: Last rotated at 2026-09-06 16:00815gitlab # [ 8.787383] logrotate[565]: log does not need rotating (log has already been rotated)816gitlab # [ 8.789030] logrotate[565]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed817gitlab: Guest shell says: b'Spawning backdoor root shell...\n'818gitlab # [ 8.796275] logrotate[565]: considering log /var/log/wtmp819gitlab # [ 8.797531] logrotate[565]: Creating new state820gitlab: connected to guest root shell821gitlab # [ 8.798176] logrotate[565]: Now: 2026-09-06 16:30822gitlab: (connecting took 9.25 seconds)823gitlab # [ 8.798844] logrotate[565]: Last rotated at 2026-09-06 16:00824gitlab: (finished: waiting for the VM to finish booting, in 9.25 seconds)825gitlab # [ 8.799634] logrotate[565]: log does not need rotating (log has already been rotated)826gitlab # [ 8.801332] logrotate[565]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed827gitlab # [ 8.803267] logrotate[565]: switching euid from 0 to 165 and egid from 0 to 165 (pid 565)828gitlab # [ 8.804515] logrotate[565]: considering log /var/gitlab/state/log/*.log829gitlab # [ 8.805603] logrotate[565]: log /var/gitlab/state/log/*.log does not exist -- skipping830gitlab # [ 8.806706] logrotate[565]: Creating new state831gitlab # [ 8.807390] logrotate[565]: switching euid from 165 to 0 and egid from 165 to 0 (pid 565)832gitlab # [ 8.810826] logrotate[565]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed833gitlab # [ 8.813036] logrotate[565]: switching euid from 0 to 60 and egid from 0 to 60 (pid 565)834gitlab # [ 8.814274] logrotate[565]: considering log /var/log/nginx/*.log835gitlab # [ 8.818822] logrotate[565]: log /var/log/nginx/*.log does not exist -- skipping836gitlab # [ 8.820184] logrotate[565]: Creating new state837gitlab # [ 8.820942] logrotate[565]: not running postrotate script, since no logs were rotated838gitlab # [ 8.822139] logrotate[565]: switching euid from 60 to 0 and egid from 60 to 0 (pid 565)839gitlab # [ 8.823437] systemd[1]: Starting Post-Boot Actions...840gitlab # [ 8.826585] systemd[1]: Started Reset console on configuration changes.841gitlab # [ 8.829012] nsncd[566]: Sep 06 16:30:56.744 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"842gitlab # [ 8.832163] systemd[1]: Starting resolvconf update...843gitlab # [ 8.835719] systemd[1]: Started Name Service Cache Daemon (nsncd).844gitlab # [ 8.837282] systemd[1]: Finished Logrotate configuration check.845gitlab # [ 8.838414] systemd[1]: Finished Post-Boot Actions.846gitlab # [ 8.839445] systemd[1]: Finished Import lastlog data into lastlog2 database.847gitlab # [ 8.848446] systemd[1]: Reached target Host and Network Name Lookups.848gitlab # [ 8.849504] systemd[1]: Reached target User and Group Name Lookups.849gitlab # [ 8.850563] systemd[1]: Starting User Login Management...850gitlab # [ 9.017358] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3851gitlab # [ 8.927586] systemd-logind[609]: New seat seat0.852gitlab # [ 9.018879] systemd[1]: Stopped target Host and Network Name Lookups.853gitlab # [ 9.031303] systemd[1]: Stopping Host and Network Name Lookups...854gitlab # [ 9.032528] systemd[1]: Stopped target User and Group Name Lookups.855gitlab # [ 9.033482] systemd[1]: Stopping User and Group Name Lookups...856gitlab # [ 9.038219] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...857gitlab # [ 9.039233] systemd[1]: nscd.service: Deactivated successfully.858gitlab # [ 9.040146] systemd[1]: Stopped Name Service Cache Daemon (nsncd).859gitlab # [ 9.041035] systemd[1]: nscd.service: Consumed 30ms CPU time over 212ms wall clock time, 7.6M memory peak, 5.8M read from disk.860gitlab # [ 9.042662] systemd[1]: Finished resolvconf update.861gitlab # [ 9.043408] systemd[1]: Reached target Preparation for Network.862gitlab # [ 9.044277] systemd[1]: Starting DHCP Client...863gitlab # [ 9.045006] systemd-logind[609]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)864gitlab # [ 9.046266] systemd-logind[609]: Watching system buttons on /dev/input/event0 (gpio-keys)865gitlab # [ 9.081339] systemd[1]: Starting Address configuration of eth1...866gitlab # [ 9.103390] systemd[1]: Starting Extra networking commands....867gitlab # [ 9.113284] systemd[1]: Starting Name Service Cache Daemon (nsncd)...868gitlab # [ 9.114334] systemd[1]: Started User Login Management.869gitlab # [ 9.151066] network-addresses-eth1-start[674]: adding address 192.168.1.1/24... done870gitlab # [ 9.152376] systemd[1]: Starting linger-users.service...871gitlab # [ 9.157101] systemd-logind[609]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)872gitlab # [ 9.159889] systemd[1]: Starting Virtual Console Setup...873gitlab # [ 9.163764] network-addresses-eth1-start[674]: adding address 2001:db8:1::1/64... done874gitlab # [ 9.171532] nsncd[677]: Sep 06 16:30:57.115 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"875gitlab # [ 9.174968] systemd[1]: Started Name Service Cache Daemon (nsncd).876gitlab # [ 9.175976] systemd[1]: Reached target Host and Network Name Lookups.877gitlab # [ 9.177837] systemd[1]: Reached target User and Group Name Lookups.878gitlab # [ 9.186866] systemd[1]: Finished Address configuration of eth1.879gitlab # [ 9.190233] dhcpcd[679]: dhcpcd-10.3.2 starting880gitlab # [ 9.193204] dhcpcd[729]: dev: loaded udev881gitlab # [ 9.200088] systemd[1]: linger-users.service: Deactivated successfully.882gitlab # [ 9.201852] systemd[1]: Finished linger-users.service.883gitlab # [ 9.210777] 8021q: 802.1Q VLAN Support v1.8884gitlab # [ 9.211165] 8021q: adding VLAN 0 to HW filter on device eth1885gitlab # [ 9.224103] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.886gitlab # [ 9.240568] systemd-vconsole-setup[701]: Configuration of first virtual console was skipped, ignoring remaining ones.887gitlab # [ 9.244449] systemd[1]: Finished Virtual Console Setup.888gitlab # [ 9.248124] systemd[1]: Finished Extra networking commands..889gitlab # [ 9.252546] cfg80211: Loading compiled-in X.509 certificates for regulatory database890gitlab # [ 9.253280] systemd[1]: Reached target Network.891gitlab # [ 9.256985] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'892gitlab # [ 9.257502] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'893gitlab # [ 9.259168] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2894gitlab # [ 9.259494] cfg80211: failed to load regulatory.db895gitlab # [ 9.260988] systemd[1]: Starting gitlab-workhorse.service...896gitlab # [ 9.266643] systemd[1]: Starting Nginx Web Server...897gitlab # [ 9.271070] systemd[1]: Starting PostgreSQL Server...898gitlab # [ 9.276172] systemd[1]: Starting Redis Server - redis-gitlab...899gitlab # [ 9.282690] systemd[1]: Starting Permit User Sessions...900gitlab # [ 9.300255] 8021q: adding VLAN 0 to HW filter on device eth0901gitlab # [ 9.298546] dhcpcd[729]: eth0: waiting for carrier902gitlab # [ 9.300386] dhcpcd[729]: eth0: carrier acquired903gitlab # [ 9.306879] dhcpcd[729]: DUID 00:01:00:01:32:30:51:c1:52:54:00:12:34:56904gitlab # [ 9.311924] dhcpcd[729]: eth0: IAID 00:12:34:56905gitlab # [ 9.312848] dhcpcd[729]: eth0: adding address fe80::5054:ff:fe12:3456906gitlab # [ 9.314304] systemd[1]: Finished Permit User Sessions.907gitlab # [ 9.327699] systemd[1]: Started Getty on tty1.908gitlab # [ 9.328694] systemd[1]: Reached target Login Prompts.909gitlab # [ 9.331917] dhcpcd[729]: eth0: soliciting a DHCP lease910gitlab # [ 9.347932] dhcpcd[729]: eth0: offered 10.0.2.15 from 10.0.2.2911gitlab # [ 9.360392] dhcpcd[729]: eth0: probing address 10.0.2.15/24912gitlab # [ 9.420621] nginx-pre-start[793]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok913gitlab # [ 9.422552] nginx-pre-start[793]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful914gitlab # [ 9.429454] systemd[1]: Started Nginx Web Server.915gitlab # [ 9.465735] redis-server[792]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.916gitlab # [ 9.470375] redis-server[792]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo917gitlab # [ 9.471772] redis-server[792]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=792, just started918gitlab # [ 9.475658] systemd[1]: Started Redis Server - redis-gitlab.919gitlab # [ 9.476884] redis-server[792]: Configuration loaded920gitlab # [ 9.478085] redis[792]: monotonic clock: ARM CNTVCT @ 1000 ticks/us921gitlab # [ 9.479360] redis[792]: Running mode=standalone, port=0.922gitlab # [ 9.480639] redis[792]: Server initialized923gitlab # [ 9.481498] redis[792]: Ready to accept connections unix924gitlab # [ 9.482549] redis[792]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.925gitlab # [ 9.485457] postgresql-pre-start[795]: The files belonging to this database system will be owned by user "postgres".926gitlab # [ 9.487502] postgresql-pre-start[795]: This user must also own the server process.927gitlab # [ 9.488963] postgresql-pre-start[795]: The database cluster will be initialized with locale "en_US.UTF-8".928gitlab # [ 9.490415] postgresql-pre-start[795]: The default database encoding has accordingly been set to "UTF8".929gitlab # [ 9.491966] postgresql-pre-start[795]: The default text search configuration will be set to "english".930gitlab # [ 9.493578] postgresql-pre-start[795]: Data page checksums are enabled.931gitlab # [ 9.494683] postgresql-pre-start[795]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok932gitlab # [ 9.496385] postgresql-pre-start[795]: creating subdirectories ... ok933gitlab # [ 9.497474] postgresql-pre-start[795]: selecting dynamic shared memory implementation ... posix934gitlab # [ 9.535914] postgresql-pre-start[795]: selecting default "max_connections" ... 100935gitlab # [ 9.600614] postgresql-pre-start[795]: selecting default "shared_buffers" ... 128MB936gitlab # [ 9.927684] systemd[1]: Started gitlab-workhorse.service.937gitlab # [ 10.104313] postgresql-pre-start[795]: selecting default time zone ... UTC938gitlab # [ 10.107403] postgresql-pre-start[795]: creating configuration files ... ok939gitlab # [ 10.133485] gitlab-workhorse[823]: time="2026-09-06T16:30:58Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4940gitlab # [ 10.135536] gitlab-workhorse[823]: time="2026-09-06T16:30:58Z" level=info msg="Using redis/go-redis"941gitlab # [ 10.142354] gitlab-workhorse[823]: time="2026-09-06T16:30:58Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix942gitlab # [ 10.343535] postgresql-pre-start[795]: running bootstrap script ... ok943gitlab # [ 11.001802] postgresql-pre-start[795]: performing post-bootstrap initialization ... ok944gitlab # [ 11.368003] postgresql-pre-start[795]: syncing data to disk ... ok945gitlab # [ 11.370970] postgresql-pre-start[795]: initdb: warning: enabling "trust" authentication for local connections946gitlab # [ 11.375231] postgresql-pre-start[795]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.947gitlab # [ 11.381916] postgresql-pre-start[795]: Success. You can now start the database server using:948gitlab # [ 11.385622] postgresql-pre-start[795]: pg_ctl -D /var/lib/postgresql/18 -l logfile start949gitlab # [ 11.586442] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit950gitlab # [ 11.592576] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432951gitlab # [ 11.596132] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432952gitlab # [ 11.599393] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"953gitlab # [ 11.605735] postgres[868]: [868] LOG: database system was shut down at 2026-09-06 16:30:58 GMT954gitlab # [ 11.614938] postgres[854]: [854] LOG: database system is ready to accept connections955gitlab # [ 11.622187] systemd[1]: Started PostgreSQL Server.956gitlab # [ 11.625515] dhcpcd[729]: eth0: soliciting an IPv6 router957gitlab # [ 11.627880] dhcpcd[729]: eth0: Router Advertisement from fe80::2958gitlab # [ 11.630841] systemd[1]: Starting PostgreSQL Setup Scripts...959gitlab # [ 11.633486] dhcpcd[729]: eth0: adding address fec0::5054:ff:fe12:3456/64960gitlab # [ 11.636322] dhcpcd[729]: eth0: adding route to fec0::/64961gitlab # [ 11.638795] dhcpcd[729]: eth0: adding default route via fe80::2962gitlab # [ 11.729552] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[861]: mkdir -p /var/gitlab/state/home/.ssh: OK963gitlab # [ 11.734660] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[861]: chmod 700 /var/gitlab/state/home/.ssh: OK964gitlab # [ 11.836906] systemd[1]: Finished gitlab-config.service.965gitlab # [ 11.845757] systemd[1]: Started gitaly.service.966gitlab # [ 11.976804] postgresql-setup-start[885]: CREATE DATABASE967gitlab # [ 12.026964] postgresql-setup-start[898]: CREATE ROLE968gitlab # [ 12.056900] postgresql-setup-start[905]: ALTER DATABASE969gitlab # [ 12.119965] postgresql-setup-start[919]: CREATE ROLE970gitlab # [ 12.126559] systemd[1]: Finished PostgreSQL Setup Scripts.971gitlab # [ 12.128369] systemd[1]: Reached target PostgreSQL.972gitlab # [ 12.133597] systemd[1]: Starting gitlab-db-config.service...973gitlab # [ 12.138812] systemd[1]: Starting gitlab-postgresql.service...974gitlab # [ 12.351841] gitlab-postgresql-start[931]: CREATE DATABASE975gitlab # [ 12.428313] gitlab-postgresql-start[936]: CREATE EXTENSION976gitlab # [ 12.460796] gitaly[893]: time="2026-09-06T16:31:00.402Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=893977gitlab # [ 12.466872] gitaly[893]: time="2026-09-06T16:31:00.402Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=893978gitlab # [ 12.469540] gitaly[893]: time="2026-09-06T16:31:00.402Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=893979gitlab # [ 12.472937] gitaly[893]: time="2026-09-06T16:31:00.402Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=893980gitlab # [ 12.475783] gitaly[893]: time="2026-09-06T16:31:00.402Z" level=info msg="Starting Gitaly" pid=893 version=981gitlab # [ 12.587403] gitlab-postgresql-start[938]: CREATE EXTENSION982gitlab # [ 12.600141] systemd[1]: Finished gitlab-postgresql.service.983gitlab # [ 12.696462] gitaly[893]: time="2026-09-06T16:31:00.640Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=231 pid=893984gitlab # [ 12.698769] gitaly[893]: time="2026-09-06T16:31:00.643Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=893985gitlab # [ 12.701088] gitaly[893]: time="2026-09-06T16:31:00.645Z" level=info msg="finished initializing command factory" duration_ms=0 pid=893986gitlab # [ 12.702993] gitaly[893]: time="2026-09-06T16:31:00.647Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-893/git-exec-3231184438.d/git pid=893987gitlab # [ 12.710693] gitaly[893]: time="2026-09-06T16:31:00.655Z" level=info msg="finished detecting git version" duration_ms=5 pid=893988gitlab # [ 12.712568] gitaly[893]: time="2026-09-06T16:31:00.656Z" level=info msg="using Git version" pid=893 version=2.53.g989gitlab # [ 12.714189] gitaly[893]: time="2026-09-06T16:31:00.657Z" level=info msg="clearing disk cache object folder" pid=893 storage=default990gitlab # [ 12.715902] gitaly[893]: time="2026-09-06T16:31:00.657Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2815198647 pid=893 storage=default991gitlab # [ 12.718713] gitaly[893]: time="2026-09-06T16:31:00.657Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2815198647 pid=893 storage=default992gitlab # [ 12.721756] gitaly[893]: time="2026-09-06T16:31:00.657Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=893993gitlab # [ 12.724469] gitaly[893]: time="2026-09-06T16:31:00.657Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=893994gitlab # [ 12.727456] gitaly[893]: time="2026-09-06T16:31:00.657Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2815198647 clear_duration_ms=0 pid=893 storage=default995gitlab # [ 12.730897] gitaly[893]: time="2026-09-06T16:31:00.674Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=893996gitlab # [ 12.733326] gitaly[893]: time="2026-09-06T16:31:00.674Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=893 schema=unix997gitlab # [ 12.735438] gitaly[893]: time="2026-09-06T16:31:00.674Z" level=info msg="listening at address" address=/run/gitaly/gitaly-893/sock.d/intern pid=893 schema=unix998gitlab # [ 12.737571] gitaly[893]: time="2026-09-06T16:31:00.675Z" level=info msg="finished tempdir cleaner walk" pid=893 storage=default time_ms=0999gitlab # [ 12.739401] gitaly[893]: time="2026-09-06T16:31:00.675Z" level=info msg="counting repositories" pid=8931000gitlab # [ 12.740849] gitaly[893]: time="2026-09-06T16:31:00.675Z" level=info msg="starting to count repositories" pid=893 storage_path=/var/gitlab/state/repositories1001gitlab # [ 12.742843] gitaly[893]: time="2026-09-06T16:31:00.675Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=893 storage_path=/var/gitlab/state/repositories1002gitlab # [ 12.745187] gitaly[893]: time="2026-09-06T16:31:00.675Z" level=info msg="completed counting all repositories" counting_duration=0 pid=8931003gitlab # [ 12.746960] gitaly[893]: time="2026-09-06T16:31:00.676Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=8931004gitlab # [ 12.748778] gitaly[893]: time="2026-09-06T16:31:00.677Z" level=info msg="Started Gitaly" duration_ms=274 pid=8931005gitlab # [ 12.750275] gitaly[893]: time="2026-09-06T16:31:00.677Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=8931006gitlab # [ 12.752041] gitaly[893]: time="2026-09-06T16:31:00.677Z" level=info msg="maintenance: daily scheduled" pid=893 scheduled="2026-09-07 12:00:00 +0000 UTC"1007gitlab # [ 13.582539] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1008gitlab # [ 13.582942] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1009gitlab # [ 13.598383] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1010gitlab # [ 14.170773] dhcpcd[729]: eth0: leased 10.0.2.15 for 86400 seconds1011gitlab # [ 14.173986] dhcpcd[729]: eth0: adding route to 10.0.2.0/241012gitlab # [ 14.176702] dhcpcd[729]: eth0: adding default route via 10.0.2.21013gitlab # [ 14.362822] systemd[1]: Started DHCP Client.1014gitlab # [ 14.364513] systemd[1]: Reached target Network is Online.1015gitlab # [ 16.274590] gitaly[893]: time="2026-09-06T16:31:04.217Z" level=info msg="License database preloaded" duration_ms=3814 pid=8931016gitlab # [ 62.107489] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: W, [2026-09-06T16:31:50.050995 #926] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1017gitlab # [ 65.608328] postgres[1465]: [1465] ERROR: relation "feature_gates" does not exist at character 5231018gitlab # [ 65.609746] postgres[1465]: [1465] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1019gitlab # [ 65.611074] postgres[1465]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1020gitlab # [ 65.612509] postgres[1465]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1021gitlab # [ 65.613755] postgres[1465]: attidentity AS identity,1022gitlab # [ 65.614578] postgres[1465]: attgenerated as attgenerated1023gitlab # [ 65.615455] postgres[1465]: FROM pg_attribute a1024gitlab # [ 65.616197] postgres[1465]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1025gitlab # [ 65.617494] postgres[1465]: LEFT JOIN pg_type t ON a.atttypid = t.oid1026gitlab # [ 65.618460] postgres[1465]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1027gitlab # [ 65.619936] postgres[1465]: WHERE a.attrelid = '"feature_gates"'::regclass1028gitlab # [ 65.620976] postgres[1465]: AND a.attnum > 0 AND NOT a.attisdropped1029gitlab # [ 65.621925] postgres[1465]: ORDER BY a.attnum1030gitlab # [ 65.622594] postgres[1465]:1031gitlab # [ 70.226148] gitlab-workhorse[823]: time="2026-09-06T16:31:58Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1032gitlab # [ 71.228336] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1525]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1033gitlab # [ 71.228808] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1525]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1034gitlab # [ 80.233281] gitlab-workhorse[823]: time="2026-09-06T16:32:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1035gitlab # [ 90.235116] gitlab-workhorse[823]: time="2026-09-06T16:32:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1036gitlab # [ 100.242033] gitlab-workhorse[823]: time="2026-09-06T16:32:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1037gitlab # [ 101.123071] postgres[1646]: [1646] ERROR: relation "table_without_models" does not exist1038gitlab # [ 101.123475] postgres[1646]: [1646] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1039gitlab # [ 110.250375] gitlab-workhorse[823]: time="2026-09-06T16:32:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1040gitlab # [ 120.261085] gitlab-workhorse[823]: time="2026-09-06T16:32:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1041gitlab # [ 130.265167] gitlab-workhorse[823]: time="2026-09-06T16:32:58Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1042gitlab # [ 135.211261] systemd[1]: Finished gitlab-db-config.service.1043gitlab # [ 135.218325] systemd[1]: Started gitlab-sidekiq.service.1044gitlab # [ 135.224114] systemd[1]: Starting gitlab.service...1045gitlab # [ 135.823506] bundle[2067]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1046gitlab # [ 135.830334] bundle[2067]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1047gitlab # [ 135.836568] bundle[2067]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1048gitlab # [ 140.266951] gitlab-workhorse[823]: time="2026-09-06T16:33:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1049gitlab # [ 141.115743] sidekiq-cluster[2066]: {"severity":"INFO","time":"2026-09-06T16:33:09.019Z","message":"Starting cluster with 1 processes"}1050gitlab # [ 141.128533] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.033Z","pid":2067,"message":"* Enabling systemd notification integration"}1051gitlab # [ 141.133586] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.071Z","pid":2067,"message":"Puma starting in cluster mode..."}1052gitlab # [ 141.138051] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.071Z","pid":2067,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1053gitlab # [ 141.142868] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1054gitlab # [ 141.148798] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Min threads: 0"}1055gitlab # [ 141.152666] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Max threads: 4"}1056gitlab # [ 141.156506] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Environment: production"}1057gitlab # [ 141.160653] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Master PID: 2067"}1058gitlab # [ 141.164644] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Workers: 2"}1059gitlab # [ 141.168464] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1060gitlab # [ 141.172989] bundle[2067]: {"timestamp":"2026-09-06T16:33:09.072Z","pid":2067,"message":"* Preloading application"}1061gitlab # [ 150.274209] gitlab-workhorse[823]: time="2026-09-06T16:33:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1062gitlab # [ 160.275394] gitlab-workhorse[823]: time="2026-09-06T16:33:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1063gitlab # [ 170.282624] gitlab-workhorse[823]: time="2026-09-06T16:33:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1064gitlab # [ 180.290372] gitlab-workhorse[823]: time="2026-09-06T16:33:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1065gitlab # [ 184.815153] bundle[2067]: W, [2026-09-06T16:33:52.758131 #2067] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1066gitlab # [ 190.298821] gitlab-workhorse[823]: time="2026-09-06T16:33:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1067gitlab # [ 191.295231] sidekiq-cluster[2115]: 2026-09-06T16:33:59.238Z pid=2115 tid=xrf INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2115, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>432869376}1068gitlab # [ 191.683860] sidekiq-cluster[2115]: W, [2026-09-06T16:33:59.627913 #2115] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1069gitlab # [ 191.756096] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:33:59.699Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1070gitlab # [ 195.911090] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.854Z","message":"GitLab reliable fetch activated!"}1071gitlab # [ 195.911577] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.854Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1072gitlab # [ 195.958266] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.902Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1073gitlab # [ 195.965583] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.904Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1074gitlab # [ 195.974142] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.907Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1075gitlab # [ 195.981092] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.909Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1076gitlab # [ 195.992403] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.912Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1077gitlab # [ 195.998247] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.915Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1078gitlab # [ 196.003636] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.917Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1079gitlab # [ 196.010699] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.919Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1080gitlab # [ 196.015491] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.921Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1081gitlab # [ 196.019932] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.922Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1082gitlab # [ 196.024638] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.924Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1083gitlab # [ 196.028720] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.927Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1084gitlab # [ 196.032493] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.929Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1085gitlab # [ 196.036543] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.931Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1086gitlab # [ 196.040505] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.933Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1087gitlab # [ 196.044373] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.935Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1088gitlab # [ 196.047834] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.938Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1089gitlab # [ 196.051280] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.941Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1090gitlab # [ 196.054809] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.944Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1091gitlab # [ 196.057951] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.947Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1092gitlab # [ 196.061255] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.949Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1093gitlab # [ 196.064583] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.952Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1094gitlab # [ 196.067393] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.955Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1095gitlab # [ 196.070341] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.956Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1096gitlab # [ 196.073167] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.958Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1097gitlab # [ 196.075715] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.961Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1098gitlab # [ 196.078295] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.964Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1099gitlab # [ 196.080786] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.967Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1100gitlab # [ 196.083255] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.969Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1101gitlab # [ 196.085832] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.972Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1102gitlab # [ 196.088057] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.974Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1103gitlab # [ 196.090525] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.977Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1104gitlab # [ 196.092816] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.981Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1105gitlab # [ 196.095150] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.983Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1106gitlab # [ 196.097528] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.985Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1107gitlab # [ 196.099737] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.987Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1108gitlab # [ 196.102129] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.989Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1109gitlab # [ 196.104201] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.991Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1110gitlab # [ 196.106253] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.993Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1111gitlab # [ 196.108306] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.995Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1112gitlab # [ 196.110477] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.997Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1113gitlab # [ 196.112675] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:03.999Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1114gitlab # [ 196.114843] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.001Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1115gitlab # [ 196.116963] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.003Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1116gitlab # [ 196.119137] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.006Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1117gitlab # [ 196.121519] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.008Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1118gitlab # [ 196.123651] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.010Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1119gitlab # [ 196.125946] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.012Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1120gitlab # [ 196.128059] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.014Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1121gitlab # [ 196.130179] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.016Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1122gitlab # [ 196.132470] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.019Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1123gitlab # [ 196.135666] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.021Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1124gitlab # [ 196.137929] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.033Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1125gitlab # [ 196.140398] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.036Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1126gitlab # [ 196.142638] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.039Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1127gitlab # [ 196.144929] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.041Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1128gitlab # [ 196.147174] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.043Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1129gitlab # [ 196.149644] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.046Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1130gitlab # [ 196.151926] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.048Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1131gitlab # [ 196.154418] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.050Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1132gitlab # [ 196.156576] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.053Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1133gitlab # [ 196.158775] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.055Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1134gitlab # [ 196.161069] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.057Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1135gitlab # [ 196.163311] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.060Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1136gitlab # [ 196.165743] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.063Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1137gitlab # [ 196.168099] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.065Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1138gitlab # [ 196.170279] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.067Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1139gitlab # [ 196.172567] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.069Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1140gitlab # [ 196.174954] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.071Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1141gitlab # [ 196.177436] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.073Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1142gitlab # [ 196.179745] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.075Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1143gitlab # [ 196.182192] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.077Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1144gitlab # [ 196.184543] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.079Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1145gitlab # [ 196.186980] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.081Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1146gitlab # [ 196.189493] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.083Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1147gitlab # [ 196.191909] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.085Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1148gitlab # [ 196.194386] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.087Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1149gitlab # [ 196.197085] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.089Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1150gitlab # [ 196.199373] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.091Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1151gitlab # [ 196.201694] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.093Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1152gitlab # [ 196.204127] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.095Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1153gitlab # [ 196.206624] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.097Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1154gitlab # [ 196.209076] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.100Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1155gitlab # [ 196.211367] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.102Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1156gitlab # [ 196.213935] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.106Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1157gitlab # [ 196.216269] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.109Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1158gitlab # [ 196.218712] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.112Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1159gitlab # [ 196.221155] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.115Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1160gitlab # [ 196.223554] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.118Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1161gitlab # [ 196.226075] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.120Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1162gitlab # [ 196.228637] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.123Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1163gitlab # [ 196.231189] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.125Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1164gitlab # [ 196.233712] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.127Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1165gitlab # [ 196.236252] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.129Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1166gitlab # [ 196.238332] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.132Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1167gitlab # [ 196.240834] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.135Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1168gitlab # [ 196.243175] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.139Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1169gitlab # [ 196.245624] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.141Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1170gitlab # [ 196.247904] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.155Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1171gitlab # [ 196.250367] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.163Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1172gitlab # [ 196.252651] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.169Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1173gitlab # [ 196.254986] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.173Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1174gitlab # [ 196.257182] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.176Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1175gitlab # [ 196.259421] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.179Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1176gitlab # [ 196.261990] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.183Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1177gitlab # [ 196.264452] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.185Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1178gitlab # [ 196.266642] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.187Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1179gitlab # [ 196.301960] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.246Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1180gitlab # [ 196.306151] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:04.250Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1181gitlab # [ 200.306409] gitlab-workhorse[823]: time="2026-09-06T16:34:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1182gitlab # [ 210.316618] gitlab-workhorse[823]: time="2026-09-06T16:34:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1183gitlab # [ 220.327466] gitlab-workhorse[823]: time="2026-09-06T16:34:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1184gitlab # [ 230.339581] gitlab-workhorse[823]: time="2026-09-06T16:34:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1185gitlab # [ 239.836678] postgres[2943]: [2943] ERROR: relation "table_without_models" does not exist1186gitlab # [ 239.837128] postgres[2943]: [2943] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1187gitlab # [ 240.346402] gitlab-workhorse[823]: time="2026-09-06T16:34:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1188gitlab # [ 240.796590] postgres[2954]: [2954] ERROR: relation "table_without_models" does not exist1189gitlab # [ 240.799824] postgres[2954]: [2954] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1190gitlab # [ 245.450689] bundle[2067]: {"timestamp":"2026-09-06T16:34:53.393Z","pid":2067,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1191gitlab # [ 245.451302] bundle[2067]: {"timestamp":"2026-09-06T16:34:53.393Z","pid":2067,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1192gitlab # [ 245.451445] bundle[2067]: {"timestamp":"2026-09-06T16:34:53.394Z","pid":2067,"message":"! #\u003cThread:0x0000ff15cb037e48 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1193gitlab # [ 245.451571] bundle[2067]: {"timestamp":"2026-09-06T16:34:53.394Z","pid":2067,"message":"! #\u003cThread:0x0000ff15cb24a0c8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1194gitlab # [ 245.451729] bundle[2067]: {"timestamp":"2026-09-06T16:34:53.394Z","pid":2067,"message":"Use Ctrl-C to stop"}1195gitlab # [ 245.451844] bundle[2067]: {"timestamp":"2026-09-06T16:34:53.395Z","pid":2067,"message":"Sending status to systemd every 1.0 sec"}1196gitlab # [ 246.134237] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:54.068Z","message":"Booted Rails 7.2.3.1 application in production environment"}1197gitlab # [ 246.134520] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:54.076Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1198gitlab # [ 246.134606] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:54.076Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1199gitlab # [ 246.134717] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:54.077Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1200gitlab # [ 246.137855] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:34:54.082Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1201gitlab # [ 246.560436] bundle[2067]: {"timestamp":"2026-09-06T16:34:54.500Z","pid":2067,"message":"- Worker 0 (PID: 3044) booted in 1.04s, phase: 0"}1202gitlab # [ 246.615219] bundle[2067]: {"timestamp":"2026-09-06T16:34:54.559Z","pid":2067,"message":"- Worker 1 (PID: 3048) booted in 1.07s, phase: 0"}1203gitlab # [ 246.625392] systemd[1]: Started gitlab.service.1204gitlab # [ 246.627588] systemd[1]: Reached target Common target for all GitLab services..1205gitlab # [ 246.640191] systemd[1]: Starting setup-gitlab.service...1206gitlab # [ 246.706840] setup-gitlab-start[3083]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1207gitlab # [ 252.222525] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:00.163Z","message":"Cleaning working queues"}1208gitlab # [ 252.230990] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:00.172Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c5c9867604ee23f2b95f5270","created_at":"2026-09-06T16:33:02.552Z","trace_propagation_headers":{"sentry-trace":"8c5e38a370c04520b01af112b2083405-42623d7885544204","baggage":"sentry-trace_id=8c5e38a370c04520b01af112b2083405,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d810839b89825e43660481c3e8811fb6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:02.566Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c5c9867604ee23f2b95f5270: start","job_status":"start","queue_duration_s":117.605898,"scheduling_latency_s":117.60601}1209gitlab # [ 253.494033] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:01.432Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"c5c9867604ee23f2b95f5270","created_at":"2026-09-06T16:33:02.552Z","trace_propagation_headers":{"sentry-trace":"8c5e38a370c04520b01af112b2083405-42623d7885544204","baggage":"sentry-trace_id=8c5e38a370c04520b01af112b2083405,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"d810839b89825e43660481c3e8811fb6","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:33:02.566Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-c5c9867604ee23f2b95f5270: done: 1.260522 sec","job_status":"done","queue_duration_s":117.605898,"scheduling_latency_s":117.60601,"redis_calls":24,"redis_duration_s":0.030254,"redis_read_bytes":912,"redis_write_bytes":2622,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.01349,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.01048,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006284,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":9,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.08,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.711841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.8188305,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":1.260522,"completed_at":"2026-09-06T16:35:01.432Z","load_balancing_strategy":"primary","db_duration_s":0.386005,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1210gitlab # [ 255.124856] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:35:03 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 83511211gitlab # [ 255.129661] setup-gitlab-start[3083]: + TOKEN=nixbot-test-token-of-20-chars1212gitlab # [ 255.132992] setup-gitlab-start[3083]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1213gitlab # [ 255.156891] runuser[3137]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1214gitlab # [ 255.768462] setup-gitlab-start[3138]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1215gitlab # [ 255.768999] setup-gitlab-start[3138]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1216gitlab # [ 255.771326] setup-gitlab-start[3138]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1217gitlab # [ 256.702974] setup-gitlab-start[3138]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1218gitlab # [ 256.703372] setup-gitlab-start[3138]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1219gitlab # [ 256.706133] setup-gitlab-start[3138]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1220gitlab # [ 267.035817] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:14.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"06c3697887f98b421c6ce691","created_at":"2026-09-06T16:35:14.730Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cffd5e2553afe4ff88a920a68285b9e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:14.969Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-06c3697887f98b421c6ce691: start","job_status":"start","queue_duration_s":0.008761,"scheduling_latency_s":0.00883}1221gitlab # [ 267.555294] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:15.499Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1222gitlab # [ 267.674362] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:15.617Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"06c3697887f98b421c6ce691","created_at":"2026-09-06T16:35:14.730Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cffd5e2553afe4ff88a920a68285b9e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:14.969Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-06c3697887f98b421c6ce691: done: 0.639271 sec","job_status":"done","queue_duration_s":0.008761,"scheduling_latency_s":0.00883,"redis_calls":10,"redis_duration_s":0.058099,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007146,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.041181,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.009772,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057037,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.015142,"duration_s":0.639271,"completed_at":"2026-09-06T16:35:15.617Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.200629,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1223gitlab # [ 267.828922] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:15.772Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f18c8655d834d637e0672b09","created_at":"2026-09-06T16:35:15.030Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23ba0b06d2c707965b4d08cdba13a454","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:35:15.066Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f18c8655d834d637e0672b09: start","job_status":"start","queue_duration_s":0.706377,"scheduling_latency_s":0.706422}1224gitlab # [ 268.441718] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:16.383Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f18c8655d834d637e0672b09","created_at":"2026-09-06T16:35:15.030Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23ba0b06d2c707965b4d08cdba13a454","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:35:15.066Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f18c8655d834d637e0672b09: done: 0.61113 sec","job_status":"done","queue_duration_s":0.706377,"scheduling_latency_s":0.706422,"redis_calls":10,"redis_duration_s":0.086257,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.044258,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.029232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.012767,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.076187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00303956,"duration_s":0.61113,"completed_at":"2026-09-06T16:35:16.383Z","load_balancing_strategy":"primary","db_duration_s":0.139961,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1225gitlab # [ 268.549043] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:16.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"db03139d0c9648ebbb81d461","created_at":"2026-09-06T16:35:15.102Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.143Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-db03139d0c9648ebbb81d461: start","job_status":"start","queue_duration_s":1.348499,"scheduling_latency_s":1.348613}1226gitlab # [ 269.159187] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:17.102Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"db03139d0c9648ebbb81d461","created_at":"2026-09-06T16:35:15.102Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.143Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-db03139d0c9648ebbb81d461: done: 0.609739 sec","job_status":"done","queue_duration_s":1.348499,"scheduling_latency_s":1.348613,"redis_calls":19,"redis_duration_s":0.140584,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.050816,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.002073,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.074888,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.012807,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.081,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.081,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00501846,"duration_s":0.609739,"completed_at":"2026-09-06T16:35:17.102Z","load_balancing_strategy":"primary","db_duration_s":0.081167,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1227gitlab # [ 269.255624] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:17.199Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0652c5dfd02c5b471c07d817","created_at":"2026-09-06T16:35:15.192Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee6455c68a1d2fa650883a2b70938f1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:35:15.275Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0652c5dfd02c5b471c07d817: start","job_status":"start","queue_duration_s":1.922992,"scheduling_latency_s":1.923063}1228gitlab # [ 269.505836] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:17.448Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1229gitlab # [ 269.607947] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:17.550Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0652c5dfd02c5b471c07d817","created_at":"2026-09-06T16:35:15.192Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ee6455c68a1d2fa650883a2b70938f1a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:35:15.275Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0652c5dfd02c5b471c07d817: done: 0.351553 sec","job_status":"done","queue_duration_s":1.922992,"scheduling_latency_s":1.923063,"redis_calls":10,"redis_duration_s":0.067324,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.026442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.028724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.012158,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00591804,"duration_s":0.351553,"completed_at":"2026-09-06T16:35:17.550Z","load_balancing_strategy":"primary","db_duration_s":0.029545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1230gitlab # [ 269.699811] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:17.643Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6ff43da9e85af88becad9d60","created_at":"2026-09-06T16:35:15.336Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"198530e957004a129d09c3520e75030c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.394Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6ff43da9e85af88becad9d60: start","job_status":"start","queue_duration_s":2.249018,"scheduling_latency_s":2.249092}1231gitlab # [ 270.148233] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:18.089Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6ff43da9e85af88becad9d60","created_at":"2026-09-06T16:35:15.336Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"198530e957004a129d09c3520e75030c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.394Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6ff43da9e85af88becad9d60: done: 0.445908 sec","job_status":"done","queue_duration_s":2.249018,"scheduling_latency_s":2.249092,"redis_calls":12,"redis_duration_s":0.06791699999999999,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.017945,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.038775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.011197,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.05,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00709904,"duration_s":0.445908,"completed_at":"2026-09-06T16:35:18.088Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1232gitlab # [ 270.235111] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:18.178Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"315f248c1bbb663a7b923ec7","created_at":"2026-09-06T16:35:15.431Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62e4dfb158ace59a43bb4ce8c907d332","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.486Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-315f248c1bbb663a7b923ec7: start","job_status":"start","queue_duration_s":2.691975,"scheduling_latency_s":2.692043}1233gitlab # [ 270.488938] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:18.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"315f248c1bbb663a7b923ec7","created_at":"2026-09-06T16:35:15.431Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62e4dfb158ace59a43bb4ce8c907d332","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.486Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-315f248c1bbb663a7b923ec7: done: 0.253697 sec","job_status":"done","queue_duration_s":2.691975,"scheduling_latency_s":2.692043,"redis_calls":10,"redis_duration_s":0.029074,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005752,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.01832,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005002,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057713,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00393898,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.253697,"completed_at":"2026-09-06T16:35:18.431Z","load_balancing_strategy":"primary","db_duration_s":0.055724,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1234gitlab # [ 270.595044] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:18.448Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4eb301a6475b573da02d317f","created_at":"2026-09-06T16:35:15.553Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1fd400f17544e1acf7637bd463e11e0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.682Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4eb301a6475b573da02d317f: start","job_status":"start","queue_duration_s":2.766864,"scheduling_latency_s":2.766928}1235gitlab # [ 270.614470] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:18.537Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1236gitlab # [ 270.642376] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:18.585Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4eb301a6475b573da02d317f","created_at":"2026-09-06T16:35:15.553Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1fd400f17544e1acf7637bd463e11e0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.682Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4eb301a6475b573da02d317f: done: 0.136619 sec","job_status":"done","queue_duration_s":2.766864,"scheduling_latency_s":2.766928,"redis_calls":10,"redis_duration_s":0.038347,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.007306,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.021306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.009735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00492244,"duration_s":0.136619,"completed_at":"2026-09-06T16:35:18.585Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1237gitlab # [ 270.720375] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:18.607Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"36fdeda4d83e59e3910dafe7","created_at":"2026-09-06T16:35:15.859Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66dfc3dc441c971afc5f5b08d7dafc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.906Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-36fdeda4d83e59e3910dafe7: start","job_status":"start","queue_duration_s":2.701046,"scheduling_latency_s":2.701112}1238gitlab # [ 271.348185] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:19.291Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"36fdeda4d83e59e3910dafe7","created_at":"2026-09-06T16:35:15.859Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66dfc3dc441c971afc5f5b08d7dafc1f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:15.906Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-36fdeda4d83e59e3910dafe7: done: 0.684075 sec","job_status":"done","queue_duration_s":2.701046,"scheduling_latency_s":2.701112,"redis_calls":12,"redis_duration_s":0.050825999999999996,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.01,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.024897,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.015929,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.38099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00810264,"exclusive_lock_hold_duration_s":0.49249106,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.684075,"completed_at":"2026-09-06T16:35:19.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.119011,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1239gitlab # [ 271.453129] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:19.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1ab8f836b1cf84863a259e1e","created_at":"2026-09-06T16:35:15.970Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77e512ba927b348f9866624e8df46c13","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.027Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1ab8f836b1cf84863a259e1e: start","job_status":"start","queue_duration_s":3.369541,"scheduling_latency_s":3.369602}1240gitlab # [ 271.664197] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:19.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1ab8f836b1cf84863a259e1e","created_at":"2026-09-06T16:35:15.970Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77e512ba927b348f9866624e8df46c13","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.027Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1ab8f836b1cf84863a259e1e: done: 0.209424 sec","job_status":"done","queue_duration_s":3.369541,"scheduling_latency_s":3.369602,"redis_calls":12,"redis_duration_s":0.030769,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004207,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.010128,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.063523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00330992,"exclusive_lock_hold_duration_s":0.03093092,"duration_s":0.209424,"completed_at":"2026-09-06T16:35:19.605Z","load_balancing_strategy":"primary","db_duration_s":0.016907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1241gitlab # [ 271.779761] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:19.723Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0308c2ba74ef5f2b20f31de1","created_at":"2026-09-06T16:35:16.117Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c225d74bb17295ff32855521851db5ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.197Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0308c2ba74ef5f2b20f31de1: start","job_status":"start","queue_duration_s":3.525919,"scheduling_latency_s":3.525968}1242gitlab # [ 272.046017] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:19.988Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"0308c2ba74ef5f2b20f31de1","created_at":"2026-09-06T16:35:16.117Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c225d74bb17295ff32855521851db5ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.197Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-0308c2ba74ef5f2b20f31de1: done: 0.2653 sec","job_status":"done","queue_duration_s":3.525919,"scheduling_latency_s":3.525968,"redis_calls":10,"redis_duration_s":0.086469,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.061488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006058,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00457778,"duration_s":0.2653,"completed_at":"2026-09-06T16:35:19.988Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005136,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 272.128199] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0b42b5ac15350abdb5dec752","created_at":"2026-09-06T16:35:16.266Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb379afe4643053ebb0a71baeae28d79","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.352Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0b42b5ac15350abdb5dec752: start","job_status":"start","queue_duration_s":3.653102,"scheduling_latency_s":3.653166}1244gitlab # [ 272.285951] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.228Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0b42b5ac15350abdb5dec752","created_at":"2026-09-06T16:35:16.266Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb379afe4643053ebb0a71baeae28d79","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.352Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0b42b5ac15350abdb5dec752: done: 0.222534 sec","job_status":"done","queue_duration_s":3.653102,"scheduling_latency_s":3.653166,"redis_calls":13,"redis_duration_s":0.051022,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.010605,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.025933,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.014484,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04032458,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.222534,"completed_at":"2026-09-06T16:35:20.228Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019193,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1245gitlab # [ 272.387259] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3c1496fdf3098d551ad3773e","created_at":"2026-09-06T16:35:16.386Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eecc367b0c193e524cce25934b1ade30","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.538Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3c1496fdf3098d551ad3773e: start","job_status":"start","queue_duration_s":3.697791,"scheduling_latency_s":3.697856}1246gitlab # [ 272.412310] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.295Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1247gitlab # [ 272.417293] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.328Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3c1496fdf3098d551ad3773e","created_at":"2026-09-06T16:35:16.386Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eecc367b0c193e524cce25934b1ade30","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.538Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3c1496fdf3098d551ad3773e: done: 0.092087 sec","job_status":"done","queue_duration_s":3.697791,"scheduling_latency_s":3.697856,"redis_calls":10,"redis_duration_s":0.005005,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002549,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00075828,"duration_s":0.092087,"completed_at":"2026-09-06T16:35:20.328Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1248gitlab # [ 272.467606] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.337Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"1efb55bd63dab936f06e722e","created_at":"2026-09-06T16:35:16.569Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f09d5d3ce1b73e81527d8ade98579b6a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.617Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-1efb55bd63dab936f06e722e: start","job_status":"start","queue_duration_s":3.719843,"scheduling_latency_s":3.719909}1249gitlab # [ 272.572325] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.515Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"1efb55bd63dab936f06e722e","created_at":"2026-09-06T16:35:16.569Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f09d5d3ce1b73e81527d8ade98579b6a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.617Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-1efb55bd63dab936f06e722e: done: 0.178217 sec","job_status":"done","queue_duration_s":3.719843,"scheduling_latency_s":3.719909,"redis_calls":17,"redis_duration_s":0.019202,"redis_read_bytes":15,"redis_write_bytes":2007,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.007851,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1150,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.007599,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001076,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00081248,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.178217,"completed_at":"2026-09-06T16:35:20.515Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1250gitlab # [ 272.679375] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b26b3cd172e3e2bdab75944a","created_at":"2026-09-06T16:35:16.685Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8ce49f9649e882393ebc89ad1d2ad73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.777Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b26b3cd172e3e2bdab75944a: start","job_status":"start","queue_duration_s":3.84392,"scheduling_latency_s":3.843992}1251gitlab # [ 272.901064] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.844Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b26b3cd172e3e2bdab75944a","created_at":"2026-09-06T16:35:16.685Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f8ce49f9649e882393ebc89ad1d2ad73","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.777Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b26b3cd172e3e2bdab75944a: done: 0.222728 sec","job_status":"done","queue_duration_s":3.84392,"scheduling_latency_s":3.843992,"redis_calls":14,"redis_duration_s":0.007993,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002034,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002654,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00172658,"exclusive_lock_hold_duration_s":0.11682626,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.222728,"completed_at":"2026-09-06T16:35:20.844Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.036408,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1252gitlab # [ 273.012519] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.848Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"17521584de7cb3ec7014488d","created_at":"2026-09-06T16:35:16.824Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9210992687eda7f5b6e78adade957303","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.877Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-17521584de7cb3ec7014488d: start","job_status":"start","queue_duration_s":3.971058,"scheduling_latency_s":3.971108}1253gitlab # [ 273.038612] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.877Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1254gitlab # [ 273.044927] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.894Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"17521584de7cb3ec7014488d","created_at":"2026-09-06T16:35:16.824Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9210992687eda7f5b6e78adade957303","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.877Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-17521584de7cb3ec7014488d: done: 0.046049 sec","job_status":"done","queue_duration_s":3.971058,"scheduling_latency_s":3.971108,"redis_calls":10,"redis_duration_s":0.00395,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000958,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000838,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058922,"duration_s":0.046049,"completed_at":"2026-09-06T16:35:20.893Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1255gitlab # [ 273.101461] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.897Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3c74b9280189500127a16701","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3c74b9280189500127a16701: start","job_status":"start","queue_duration_s":3.911204,"scheduling_latency_s":3.911241}1256gitlab # [ 273.115281] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.934Z","message":"No existing merge request to be cleaned up."}1257gitlab # [ 273.117159] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.963Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3c74b9280189500127a16701","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3c74b9280189500127a16701: done: 0.066189 sec","job_status":"done","queue_duration_s":3.911204,"scheduling_latency_s":3.911241,"redis_calls":13,"redis_duration_s":0.004789,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000734,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002493,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046137,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051174,"duration_s":0.066189,"completed_at":"2026-09-06T16:35:20.963Z","load_balancing_strategy":"primary","db_duration_s":0.006656,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 273.160809] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.965Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9a9b861813a5e6ea497fcd5","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9a9b861813a5e6ea497fcd5: start","job_status":"start","queue_duration_s":3.979586,"scheduling_latency_s":3.979623}1259gitlab # [ 273.174181] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:20.986Z","message":"No existing merge request to be cleaned up."}1260gitlab # [ 273.176029] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.004Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e9a9b861813a5e6ea497fcd5","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e9a9b861813a5e6ea497fcd5: done: 0.039504 sec","job_status":"done","queue_duration_s":3.979586,"scheduling_latency_s":3.979623,"redis_calls":10,"redis_duration_s":0.002973,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001629,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028304,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039504,"completed_at":"2026-09-06T16:35:21.004Z","load_balancing_strategy":"primary","db_duration_s":0.002964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1261gitlab # [ 273.216758] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.007Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3b02ebe677fd932eca3cc417","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3b02ebe677fd932eca3cc417: start","job_status":"start","queue_duration_s":4.021449,"scheduling_latency_s":4.021491}1262gitlab # [ 273.230107] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.028Z","message":"No existing merge request to be cleaned up."}1263gitlab # [ 273.231944] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.043Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3b02ebe677fd932eca3cc417","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3b02ebe677fd932eca3cc417: done: 0.036113 sec","job_status":"done","queue_duration_s":4.021449,"scheduling_latency_s":4.021491,"redis_calls":10,"redis_duration_s":0.0029620000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001907,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036113,"completed_at":"2026-09-06T16:35:21.043Z","load_balancing_strategy":"primary","db_duration_s":0.003073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1264gitlab # [ 273.271475] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.045Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"56e6f02b8c03b802bf40f84c","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-56e6f02b8c03b802bf40f84c: start","job_status":"start","queue_duration_s":4.059304,"scheduling_latency_s":4.059335}1265gitlab # [ 273.284846] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.058Z","message":"No existing merge request to be cleaned up."}1266gitlab # [ 273.286695] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.069Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"56e6f02b8c03b802bf40f84c","created_at":"2026-09-06T16:35:16.832Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-fbe8ac5e325b4023","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"9a436a2bfe1ce541582b245f24531136","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:16.985Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-56e6f02b8c03b802bf40f84c: done: 0.024675 sec","job_status":"done","queue_duration_s":4.059304,"scheduling_latency_s":4.059335,"redis_calls":10,"redis_duration_s":0.001647,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024675,"completed_at":"2026-09-06T16:35:21.069Z","load_balancing_strategy":"primary","db_duration_s":0.002269,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1267gitlab # [ 273.326505] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.071Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"87fae61483f53d4be58a5144","created_at":"2026-09-06T16:35:16.950Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7eb4fa62f260fa046dbc09067e5993a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:35:17.011Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-87fae61483f53d4be58a5144: start","job_status":"start","queue_duration_s":4.060169,"scheduling_latency_s":4.060199}1268gitlab # [ 273.340660] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.087Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1269gitlab # [ 273.344167] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.096Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"87fae61483f53d4be58a5144","created_at":"2026-09-06T16:35:16.950Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d7eb4fa62f260fa046dbc09067e5993a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:35:17.011Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-87fae61483f53d4be58a5144: done: 0.02509 sec","job_status":"done","queue_duration_s":4.060169,"scheduling_latency_s":4.060199,"redis_calls":10,"redis_duration_s":0.001843,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000466,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035698,"duration_s":0.02509,"completed_at":"2026-09-06T16:35:21.096Z","load_balancing_strategy":"primary","db_duration_s":0.000575,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1270gitlab # [ 273.386852] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.098Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"90a8fdd9ca52d05958a19acc","created_at":"2026-09-06T16:35:17.085Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4dbc2489f1876b7dd57858f8813fa628","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.210Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-90a8fdd9ca52d05958a19acc: start","job_status":"start","queue_duration_s":3.887314,"scheduling_latency_s":3.887345}1271gitlab # [ 273.403861] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"90a8fdd9ca52d05958a19acc","created_at":"2026-09-06T16:35:17.085Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4dbc2489f1876b7dd57858f8813fa628","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.210Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-90a8fdd9ca52d05958a19acc: done: 0.030996 sec","job_status":"done","queue_duration_s":3.887314,"scheduling_latency_s":3.887345,"redis_calls":12,"redis_duration_s":0.002283,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000388,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00091894,"exclusive_lock_hold_duration_s":0.0047663,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.030996,"completed_at":"2026-09-06T16:35:21.129Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002175,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1272gitlab # [ 273.448503] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b8213311f7379ddac59236c8","created_at":"2026-09-06T16:35:17.240Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d095c6ce147a35eba73a88609aa5722","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.269Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b8213311f7379ddac59236c8: start","job_status":"start","queue_duration_s":3.861901,"scheduling_latency_s":3.861932}1273gitlab # [ 273.566493] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"b8213311f7379ddac59236c8","created_at":"2026-09-06T16:35:17.240Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d095c6ce147a35eba73a88609aa5722","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.269Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-b8213311f7379ddac59236c8: done: 0.378679 sec","job_status":"done","queue_duration_s":3.861901,"scheduling_latency_s":3.861932,"redis_calls":565,"redis_duration_s":0.082756,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001268,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.081134,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.335297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004174,"duration_s":0.378679,"completed_at":"2026-09-06T16:35:21.509Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1274gitlab # [ 273.657907] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.512Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a8097d6531bfd773e0f82028","created_at":"2026-09-06T16:35:17.298Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"610743336bdf28b3b27bb4c4a781efca","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.341Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a8097d6531bfd773e0f82028: start","job_status":"start","queue_duration_s":4.170519,"scheduling_latency_s":4.170551}1275gitlab # [ 273.676962] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.575Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"a8097d6531bfd773e0f82028","created_at":"2026-09-06T16:35:17.298Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"610743336bdf28b3b27bb4c4a781efca","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.341Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-a8097d6531bfd773e0f82028: done: 0.062893 sec","job_status":"done","queue_duration_s":4.170519,"scheduling_latency_s":4.170551,"redis_calls":10,"redis_duration_s":0.003932,"redis_read_bytes":9,"redis_write_bytes":1109,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000826,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002254,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007768,"duration_s":0.062893,"completed_at":"2026-09-06T16:35:21.575Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011383,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 273.720998] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:21.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"688597a84145d0e2e48ef7cc","created_at":"2026-09-06T16:35:17.408Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4802f7dd875e4b93ab98b15e5cb9a9f8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.475Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-688597a84145d0e2e48ef7cc: start","job_status":"start","queue_duration_s":4.104535,"scheduling_latency_s":4.104595}1277gitlab # [ 274.199403] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.143Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"688597a84145d0e2e48ef7cc","created_at":"2026-09-06T16:35:17.408Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4802f7dd875e4b93ab98b15e5cb9a9f8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.475Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-688597a84145d0e2e48ef7cc: done: 0.563405 sec","job_status":"done","queue_duration_s":4.104535,"scheduling_latency_s":4.104595,"redis_calls":12,"redis_duration_s":0.00493,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000952,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001988,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00199,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.061,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.474121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00138482,"exclusive_lock_hold_duration_s":0.51323674,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.563405,"completed_at":"2026-09-06T16:35:22.142Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.098062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1278gitlab # [ 274.307338] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c29c3303efcfa503e78d2a18","created_at":"2026-09-06T16:35:17.532Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3992826214e995c92ad54a0b3c6dcf36","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.689Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c29c3303efcfa503e78d2a18: start","job_status":"start","queue_duration_s":4.456007,"scheduling_latency_s":4.456037}1279gitlab # [ 274.328810] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.181Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c29c3303efcfa503e78d2a18","created_at":"2026-09-06T16:35:17.532Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3992826214e995c92ad54a0b3c6dcf36","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.689Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c29c3303efcfa503e78d2a18: done: 0.036422 sec","job_status":"done","queue_duration_s":4.456007,"scheduling_latency_s":4.456037,"redis_calls":10,"redis_duration_s":0.003332,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000717,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000697,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044934,"duration_s":0.036422,"completed_at":"2026-09-06T16:35:22.181Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1280gitlab # [ 274.378526] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.184Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b8b64f5376e73309976b9903","created_at":"2026-09-06T16:35:17.726Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.796Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b8b64f5376e73309976b9903: start","job_status":"start","queue_duration_s":4.388562,"scheduling_latency_s":4.388594}1281gitlab # [ 274.394861] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.338Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"b8b64f5376e73309976b9903","created_at":"2026-09-06T16:35:17.726Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.796Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-b8b64f5376e73309976b9903: done: 0.153824 sec","job_status":"done","queue_duration_s":4.388562,"scheduling_latency_s":4.388594,"redis_calls":71,"redis_duration_s":0.023916,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.003446,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001253,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.018281,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.128231,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004371,"duration_s":0.153824,"completed_at":"2026-09-06T16:35:22.338Z","load_balancing_strategy":"primary","db_duration_s":0.003922,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 274.439425] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6375e51609d5db232108bb8c","created_at":"2026-09-06T16:35:17.833Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db9ada238fda91283d7fb240e6ea73d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.899Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6375e51609d5db232108bb8c: start","job_status":"start","queue_duration_s":4.44169,"scheduling_latency_s":4.441722}1283gitlab # [ 274.670979] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"6375e51609d5db232108bb8c","created_at":"2026-09-06T16:35:17.833Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db9ada238fda91283d7fb240e6ea73d4","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:17.899Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-6375e51609d5db232108bb8c: done: 0.273235 sec","job_status":"done","queue_duration_s":4.44169,"scheduling_latency_s":4.441722,"redis_calls":12,"redis_duration_s":0.181515,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001273,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.082033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.098209,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0013181,"exclusive_lock_hold_duration_s":0.03466922,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.273235,"completed_at":"2026-09-06T16:35:22.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.032887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1284gitlab # [ 274.722914] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.618Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"1e056a208c113cf136201520","created_at":"2026-09-06T16:35:17.946Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d793933ba49fe718e9f08788e237384","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:18.000Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-1e056a208c113cf136201520: start","job_status":"start","queue_duration_s":4.618454,"scheduling_latency_s":4.618512}1285gitlab # [ 274.747815] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.690Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"1e056a208c113cf136201520","created_at":"2026-09-06T16:35:17.946Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d793933ba49fe718e9f08788e237384","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-06T16:35:18.000Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-1e056a208c113cf136201520: done: 0.07165 sec","job_status":"done","queue_duration_s":4.618454,"scheduling_latency_s":4.618512,"redis_calls":10,"redis_duration_s":0.004749,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001169,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002559,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063988,"duration_s":0.07165,"completed_at":"2026-09-06T16:35:22.690Z","load_balancing_strategy":"primary","db_duration_s":0.009954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 274.790563] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"363dc8ea1114e6fde12c06ca","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-363dc8ea1114e6fde12c06ca: start","job_status":"start","queue_duration_s":0.369086,"scheduling_latency_s":0.369127}1287gitlab # [ 274.803849] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"363dc8ea1114e6fde12c06ca","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-363dc8ea1114e6fde12c06ca: done: 0.04804 sec","job_status":"done","queue_duration_s":0.369086,"scheduling_latency_s":0.369127,"redis_calls":16,"redis_duration_s":0.005658,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000871,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001585,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003202,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00056744,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.04804,"completed_at":"2026-09-06T16:35:22.742Z","load_balancing_strategy":"primary","db_duration_s":0.001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 274.847502] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.744Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ed4f1b5e3ab77c842d55cba","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ed4f1b5e3ab77c842d55cba: start","job_status":"start","queue_duration_s":0.419609,"scheduling_latency_s":0.419644}1289gitlab # [ 274.860426] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ed4f1b5e3ab77c842d55cba","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ed4f1b5e3ab77c842d55cba: done: 0.030721 sec","job_status":"done","queue_duration_s":0.419609,"scheduling_latency_s":0.419644,"redis_calls":13,"redis_duration_s":0.004694,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003141,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.030721,"completed_at":"2026-09-06T16:35:22.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 274.904045] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"319f6396710b6a214059739c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-319f6396710b6a214059739c: start","job_status":"start","queue_duration_s":0.454518,"scheduling_latency_s":0.454589}1291gitlab # [ 274.917460] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"319f6396710b6a214059739c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-319f6396710b6a214059739c: done: 0.055178 sec","job_status":"done","queue_duration_s":0.454518,"scheduling_latency_s":0.454589,"redis_calls":13,"redis_duration_s":0.005034,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.055178,"completed_at":"2026-09-06T16:35:22.834Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 274.959475] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"632487cdda6f9150f5bc7535","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-632487cdda6f9150f5bc7535: start","job_status":"start","queue_duration_s":0.513074,"scheduling_latency_s":0.513125}1293gitlab # [ 274.972411] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"632487cdda6f9150f5bc7535","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-632487cdda6f9150f5bc7535: done: 0.04062 sec","job_status":"done","queue_duration_s":0.513074,"scheduling_latency_s":0.513125,"redis_calls":13,"redis_duration_s":0.004882,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.04062,"completed_at":"2026-09-06T16:35:22.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 275.014273] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f988a877ed16dd1bd831bed4","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f988a877ed16dd1bd831bed4: start","job_status":"start","queue_duration_s":0.556667,"scheduling_latency_s":0.556711}1295gitlab # [ 275.027216] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f988a877ed16dd1bd831bed4","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f988a877ed16dd1bd831bed4: done: 0.035906 sec","job_status":"done","queue_duration_s":0.556667,"scheduling_latency_s":0.556711,"redis_calls":13,"redis_duration_s":0.0053939999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035906,"completed_at":"2026-09-06T16:35:22.917Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 275.068896] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a8a3836c59d959bcb276f6c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a8a3836c59d959bcb276f6c: start","job_status":"start","queue_duration_s":0.594851,"scheduling_latency_s":0.594888}1297gitlab # [ 275.083807] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a8a3836c59d959bcb276f6c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a8a3836c59d959bcb276f6c: done: 0.02876 sec","job_status":"done","queue_duration_s":0.594851,"scheduling_latency_s":0.594888,"redis_calls":13,"redis_duration_s":0.003605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002317,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02876,"completed_at":"2026-09-06T16:35:22.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 275.126668] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79f6d28281270775c041d744","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79f6d28281270775c041d744: start","job_status":"start","queue_duration_s":0.625925,"scheduling_latency_s":0.625957}1299gitlab # [ 275.140342] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79f6d28281270775c041d744","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79f6d28281270775c041d744: done: 0.026494 sec","job_status":"done","queue_duration_s":0.625925,"scheduling_latency_s":0.625957,"redis_calls":13,"redis_duration_s":0.0036479999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002394,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016885,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026494,"completed_at":"2026-09-06T16:35:22.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 275.179457] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:22.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bc1825b326803a3aa81a6a9","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bc1825b326803a3aa81a6a9: start","job_status":"start","queue_duration_s":0.65462,"scheduling_latency_s":0.654654}1301gitlab # [ 275.191981] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bc1825b326803a3aa81a6a9","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bc1825b326803a3aa81a6a9: done: 0.027134 sec","job_status":"done","queue_duration_s":0.65462,"scheduling_latency_s":0.654654,"redis_calls":13,"redis_duration_s":0.0034929999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001279,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027134,"completed_at":"2026-09-06T16:35:23.006Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 275.232375] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"340227ad8a2ad9e966af34ad","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-340227ad8a2ad9e966af34ad: start","job_status":"start","queue_duration_s":0.684125,"scheduling_latency_s":0.684159}1303gitlab # [ 275.244860] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"340227ad8a2ad9e966af34ad","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-340227ad8a2ad9e966af34ad: done: 0.027838 sec","job_status":"done","queue_duration_s":0.684125,"scheduling_latency_s":0.684159,"redis_calls":13,"redis_duration_s":0.00413,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002791,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027838,"completed_at":"2026-09-06T16:35:23.037Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 275.284335] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.039Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b819234d22b7b505595b258","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b819234d22b7b505595b258: start","job_status":"start","queue_duration_s":0.714252,"scheduling_latency_s":0.714311}1305gitlab # [ 275.298164] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b819234d22b7b505595b258","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b819234d22b7b505595b258: done: 0.046335 sec","job_status":"done","queue_duration_s":0.714252,"scheduling_latency_s":0.714311,"redis_calls":13,"redis_duration_s":0.0059180000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001818,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.046335,"completed_at":"2026-09-06T16:35:23.085Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 275.337770] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"122ec4933146a825a5ec15cd","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-122ec4933146a825a5ec15cd: start","job_status":"start","queue_duration_s":0.764589,"scheduling_latency_s":0.764653}1307gitlab # [ 275.350818] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"122ec4933146a825a5ec15cd","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-122ec4933146a825a5ec15cd: done: 0.042034 sec","job_status":"done","queue_duration_s":0.764589,"scheduling_latency_s":0.764653,"redis_calls":13,"redis_duration_s":0.003899,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002393,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.042034,"completed_at":"2026-09-06T16:35:23.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 275.390689] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c402aa92d1dc767486824ea0","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c402aa92d1dc767486824ea0: start","job_status":"start","queue_duration_s":0.809872,"scheduling_latency_s":0.809923}1309gitlab # [ 275.403680] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c402aa92d1dc767486824ea0","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c402aa92d1dc767486824ea0: done: 0.032921 sec","job_status":"done","queue_duration_s":0.809872,"scheduling_latency_s":0.809923,"redis_calls":13,"redis_duration_s":0.003249,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001288,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032921,"completed_at":"2026-09-06T16:35:23.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 275.443523] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23e439fb7d6d0d8c5dc7b78c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23e439fb7d6d0d8c5dc7b78c: start","job_status":"start","queue_duration_s":0.84519,"scheduling_latency_s":0.845233}1311gitlab # [ 275.456331] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23e439fb7d6d0d8c5dc7b78c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23e439fb7d6d0d8c5dc7b78c: done: 0.028672 sec","job_status":"done","queue_duration_s":0.84519,"scheduling_latency_s":0.845233,"redis_calls":13,"redis_duration_s":0.002895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028672,"completed_at":"2026-09-06T16:35:23.199Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 275.496572] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e37b5ed2a2b971b6a5dad9a0","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e37b5ed2a2b971b6a5dad9a0: start","job_status":"start","queue_duration_s":0.8759,"scheduling_latency_s":0.875937}1313gitlab # [ 275.509214] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e37b5ed2a2b971b6a5dad9a0","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e37b5ed2a2b971b6a5dad9a0: done: 0.026732 sec","job_status":"done","queue_duration_s":0.8759,"scheduling_latency_s":0.875937,"redis_calls":13,"redis_duration_s":0.002859,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001709,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026732,"completed_at":"2026-09-06T16:35:23.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 275.548184] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3adebf235950697da3bb78b6","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3adebf235950697da3bb78b6: start","job_status":"start","queue_duration_s":0.904776,"scheduling_latency_s":0.904815}1315gitlab # [ 275.560762] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.256Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3adebf235950697da3bb78b6","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3adebf235950697da3bb78b6: done: 0.026843 sec","job_status":"done","queue_duration_s":0.904776,"scheduling_latency_s":0.904815,"redis_calls":13,"redis_duration_s":0.002652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001686,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026843,"completed_at":"2026-09-06T16:35:23.256Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 275.599434] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ecc9fba98434fc5b06044d","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ecc9fba98434fc5b06044d: start","job_status":"start","queue_duration_s":0.934216,"scheduling_latency_s":0.934255}1317gitlab # [ 275.612462] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78ecc9fba98434fc5b06044d","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78ecc9fba98434fc5b06044d: done: 0.026326 sec","job_status":"done","queue_duration_s":0.934216,"scheduling_latency_s":0.934255,"redis_calls":13,"redis_duration_s":0.0025599999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001546,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026326,"completed_at":"2026-09-06T16:35:23.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 275.653928] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef002059ff0846e639024f9f","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef002059ff0846e639024f9f: start","job_status":"start","queue_duration_s":0.962616,"scheduling_latency_s":0.962652}1319gitlab # [ 275.666815] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.315Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef002059ff0846e639024f9f","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef002059ff0846e639024f9f: done: 0.027317 sec","job_status":"done","queue_duration_s":0.962616,"scheduling_latency_s":0.962652,"redis_calls":13,"redis_duration_s":0.002797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001756,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018296,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027317,"completed_at":"2026-09-06T16:35:23.315Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 275.708131] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e4096faf5e541635ffb514","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e4096faf5e541635ffb514: start","job_status":"start","queue_duration_s":0.992059,"scheduling_latency_s":0.9921}1321gitlab # [ 275.720792] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"53e4096faf5e541635ffb514","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-53e4096faf5e541635ffb514: done: 0.0299 sec","job_status":"done","queue_duration_s":0.992059,"scheduling_latency_s":0.9921,"redis_calls":13,"redis_duration_s":0.0029230000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001709,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0299,"completed_at":"2026-09-06T16:35:23.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 275.761247] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d6aafce1a53874f49f91c66","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d6aafce1a53874f49f91c66: start","job_status":"start","queue_duration_s":1.024195,"scheduling_latency_s":1.024237}1323gitlab # [ 275.773672] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.378Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d6aafce1a53874f49f91c66","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d6aafce1a53874f49f91c66: done: 0.02949 sec","job_status":"done","queue_duration_s":1.024195,"scheduling_latency_s":1.024237,"redis_calls":13,"redis_duration_s":0.002698,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001681,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02949,"completed_at":"2026-09-06T16:35:23.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 275.813504] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9217507fbc1966102263c281","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9217507fbc1966102263c281: start","job_status":"start","queue_duration_s":1.056298,"scheduling_latency_s":1.056333}1325gitlab # [ 275.826102] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.408Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9217507fbc1966102263c281","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9217507fbc1966102263c281: done: 0.02664 sec","job_status":"done","queue_duration_s":1.056298,"scheduling_latency_s":1.056333,"redis_calls":13,"redis_duration_s":0.002846,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001048,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02664,"completed_at":"2026-09-06T16:35:23.408Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 275.865983] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc6bcfc456f0e3a7dbaa8b1b","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc6bcfc456f0e3a7dbaa8b1b: start","job_status":"start","queue_duration_s":1.085051,"scheduling_latency_s":1.085088}1327gitlab # [ 275.878578] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cc6bcfc456f0e3a7dbaa8b1b","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cc6bcfc456f0e3a7dbaa8b1b: done: 0.026058 sec","job_status":"done","queue_duration_s":1.085051,"scheduling_latency_s":1.085088,"redis_calls":13,"redis_duration_s":0.002765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001775,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026058,"completed_at":"2026-09-06T16:35:23.436Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 275.919774] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c53e818992a4d605d040a20c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c53e818992a4d605d040a20c: start","job_status":"start","queue_duration_s":1.11327,"scheduling_latency_s":1.113306}1329gitlab # [ 275.932056] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c53e818992a4d605d040a20c","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c53e818992a4d605d040a20c: done: 0.026659 sec","job_status":"done","queue_duration_s":1.11327,"scheduling_latency_s":1.113306,"redis_calls":13,"redis_duration_s":0.003118,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026659,"completed_at":"2026-09-06T16:35:23.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 275.971535] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fdc5c05fe53928585a909ea","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fdc5c05fe53928585a909ea: start","job_status":"start","queue_duration_s":1.141972,"scheduling_latency_s":1.142011}1331gitlab # [ 275.984334] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.493Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5fdc5c05fe53928585a909ea","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5fdc5c05fe53928585a909ea: done: 0.025821 sec","job_status":"done","queue_duration_s":1.141972,"scheduling_latency_s":1.142011,"redis_calls":13,"redis_duration_s":0.002701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001651,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016615,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025821,"completed_at":"2026-09-06T16:35:23.492Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 276.024547] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"347f3b8ee61685b2d8d50389","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-347f3b8ee61685b2d8d50389: start","job_status":"start","queue_duration_s":1.170344,"scheduling_latency_s":1.170386}1333gitlab # [ 276.037157] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"347f3b8ee61685b2d8d50389","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-347f3b8ee61685b2d8d50389: done: 0.026471 sec","job_status":"done","queue_duration_s":1.170344,"scheduling_latency_s":1.170386,"redis_calls":13,"redis_duration_s":0.002692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001587,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026471,"completed_at":"2026-09-06T16:35:23.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 276.076542] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"035c07ba84c9a01e459354fa","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-035c07ba84c9a01e459354fa: start","job_status":"start","queue_duration_s":1.198838,"scheduling_latency_s":1.198874}1335gitlab # [ 276.088903] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"035c07ba84c9a01e459354fa","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-035c07ba84c9a01e459354fa: done: 0.026565 sec","job_status":"done","queue_duration_s":1.198838,"scheduling_latency_s":1.198874,"redis_calls":13,"redis_duration_s":0.002811,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001072,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001739,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026565,"completed_at":"2026-09-06T16:35:23.550Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 276.130306] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c3892b4297394ed4410f9a9","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c3892b4297394ed4410f9a9: start","job_status":"start","queue_duration_s":1.227413,"scheduling_latency_s":1.227449}1337gitlab # [ 276.143303] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.579Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c3892b4297394ed4410f9a9","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c3892b4297394ed4410f9a9: done: 0.027114 sec","job_status":"done","queue_duration_s":1.227413,"scheduling_latency_s":1.227449,"redis_calls":13,"redis_duration_s":0.0028079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001077,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001731,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027114,"completed_at":"2026-09-06T16:35:23.579Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 276.183510] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d376810923b34d9f8d76707","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d376810923b34d9f8d76707: start","job_status":"start","queue_duration_s":1.257136,"scheduling_latency_s":1.257175}1339gitlab # [ 276.196490] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.608Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d376810923b34d9f8d76707","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d376810923b34d9f8d76707: done: 0.026548 sec","job_status":"done","queue_duration_s":1.257136,"scheduling_latency_s":1.257175,"redis_calls":13,"redis_duration_s":0.0026179999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000972,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001646,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026548,"completed_at":"2026-09-06T16:35:23.608Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 276.236353] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e1fd2106d8d5a3db1405b54","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e1fd2106d8d5a3db1405b54: start","job_status":"start","queue_duration_s":1.285721,"scheduling_latency_s":1.285756}1341gitlab # [ 276.248723] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e1fd2106d8d5a3db1405b54","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e1fd2106d8d5a3db1405b54: done: 0.026722 sec","job_status":"done","queue_duration_s":1.285721,"scheduling_latency_s":1.285756,"redis_calls":13,"redis_duration_s":0.002844,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001025,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001819,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026722,"completed_at":"2026-09-06T16:35:23.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 276.288121] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dfbadeea111e3694b5f5a0d","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dfbadeea111e3694b5f5a0d: start","job_status":"start","queue_duration_s":1.314621,"scheduling_latency_s":1.314661}1343gitlab # [ 276.300441] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dfbadeea111e3694b5f5a0d","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dfbadeea111e3694b5f5a0d: done: 0.027395 sec","job_status":"done","queue_duration_s":1.314621,"scheduling_latency_s":1.314661,"redis_calls":13,"redis_duration_s":0.0030759999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027395,"completed_at":"2026-09-06T16:35:23.667Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 276.340811] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b53e15ac3eb3696372488f75","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b53e15ac3eb3696372488f75: start","job_status":"start","queue_duration_s":1.344138,"scheduling_latency_s":1.344175}1345gitlab # [ 276.353170] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b53e15ac3eb3696372488f75","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b53e15ac3eb3696372488f75: done: 0.025949 sec","job_status":"done","queue_duration_s":1.344138,"scheduling_latency_s":1.344175,"redis_calls":13,"redis_duration_s":0.002652,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000975,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001677,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025949,"completed_at":"2026-09-06T16:35:23.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 276.393143] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b97b7616111bc2c7e0e8d41","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b97b7616111bc2c7e0e8d41: start","job_status":"start","queue_duration_s":1.37244,"scheduling_latency_s":1.372507}1347gitlab # [ 276.405487] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b97b7616111bc2c7e0e8d41","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b97b7616111bc2c7e0e8d41: done: 0.025748 sec","job_status":"done","queue_duration_s":1.37244,"scheduling_latency_s":1.372507,"redis_calls":13,"redis_duration_s":0.002673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001017,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001656,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016972,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025748,"completed_at":"2026-09-06T16:35:23.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 276.445093] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fec2008209e5ecaec5b6989a","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fec2008209e5ecaec5b6989a: start","job_status":"start","queue_duration_s":1.400271,"scheduling_latency_s":1.400308}1349gitlab # [ 276.457345] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.751Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fec2008209e5ecaec5b6989a","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fec2008209e5ecaec5b6989a: done: 0.025815 sec","job_status":"done","queue_duration_s":1.400271,"scheduling_latency_s":1.400308,"redis_calls":13,"redis_duration_s":0.002617,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001003,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001614,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025815,"completed_at":"2026-09-06T16:35:23.751Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 276.495915] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e5db0b9ed61fda7597e03af","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e5db0b9ed61fda7597e03af: start","job_status":"start","queue_duration_s":1.428129,"scheduling_latency_s":1.428163}1351gitlab # [ 276.509619] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5e5db0b9ed61fda7597e03af","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5e5db0b9ed61fda7597e03af: done: 0.026372 sec","job_status":"done","queue_duration_s":1.428129,"scheduling_latency_s":1.428163,"redis_calls":13,"redis_duration_s":0.002738,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001737,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026372,"completed_at":"2026-09-06T16:35:23.779Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 276.549544] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b22c017fc4ee111f0e157367","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b22c017fc4ee111f0e157367: start","job_status":"start","queue_duration_s":1.456458,"scheduling_latency_s":1.456532}1353gitlab # [ 276.562300] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.807Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b22c017fc4ee111f0e157367","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b22c017fc4ee111f0e157367: done: 0.025478 sec","job_status":"done","queue_duration_s":1.456458,"scheduling_latency_s":1.456532,"redis_calls":13,"redis_duration_s":0.002597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00095,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001647,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016483,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025478,"completed_at":"2026-09-06T16:35:23.807Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 276.601019] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.809Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6232ea288a09a42ef1695870","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6232ea288a09a42ef1695870: start","job_status":"start","queue_duration_s":1.484269,"scheduling_latency_s":1.484305}1355gitlab # [ 276.613646] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6232ea288a09a42ef1695870","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6232ea288a09a42ef1695870: done: 0.026099 sec","job_status":"done","queue_duration_s":1.484269,"scheduling_latency_s":1.484305,"redis_calls":13,"redis_duration_s":0.002838,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001174,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001664,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017282,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026099,"completed_at":"2026-09-06T16:35:23.835Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 276.652447] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d48deec57f8106cefeb7e12a","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d48deec57f8106cefeb7e12a: start","job_status":"start","queue_duration_s":1.512497,"scheduling_latency_s":1.51254}1357gitlab # [ 276.664896] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d48deec57f8106cefeb7e12a","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d48deec57f8106cefeb7e12a: done: 0.026058 sec","job_status":"done","queue_duration_s":1.512497,"scheduling_latency_s":1.51254,"redis_calls":13,"redis_duration_s":0.002772,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001739,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026058,"completed_at":"2026-09-06T16:35:23.863Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 276.704782] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fee088c034e2e7d5fc36505","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fee088c034e2e7d5fc36505: start","job_status":"start","queue_duration_s":1.540548,"scheduling_latency_s":1.540587}1359gitlab # [ 276.718539] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fee088c034e2e7d5fc36505","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fee088c034e2e7d5fc36505: done: 0.02557 sec","job_status":"done","queue_duration_s":1.540548,"scheduling_latency_s":1.540587,"redis_calls":13,"redis_duration_s":0.00255,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001593,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02557,"completed_at":"2026-09-06T16:35:23.891Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 276.757972] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.893Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"266253d75788692758db5b6e","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-266253d75788692758db5b6e: start","job_status":"start","queue_duration_s":1.568087,"scheduling_latency_s":1.568122}1361gitlab # [ 276.771003] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"266253d75788692758db5b6e","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-266253d75788692758db5b6e: done: 0.028072 sec","job_status":"done","queue_duration_s":1.568087,"scheduling_latency_s":1.568122,"redis_calls":13,"redis_duration_s":0.0028079999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001822,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028072,"completed_at":"2026-09-06T16:35:23.921Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 276.810540] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4985559dc9f12be2cc2e968","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4985559dc9f12be2cc2e968: start","job_status":"start","queue_duration_s":1.598145,"scheduling_latency_s":1.598183}1363gitlab # [ 276.823486] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.948Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4985559dc9f12be2cc2e968","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4985559dc9f12be2cc2e968: done: 0.025473 sec","job_status":"done","queue_duration_s":1.598145,"scheduling_latency_s":1.598183,"redis_calls":13,"redis_duration_s":0.002676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001016,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00166,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016581,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025473,"completed_at":"2026-09-06T16:35:23.948Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 276.862774] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cff95161e021dfef3c300502","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cff95161e021dfef3c300502: start","job_status":"start","queue_duration_s":1.625903,"scheduling_latency_s":1.62594}1365gitlab # [ 276.875387] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cff95161e021dfef3c300502","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cff95161e021dfef3c300502: done: 0.026875 sec","job_status":"done","queue_duration_s":1.625903,"scheduling_latency_s":1.62594,"redis_calls":13,"redis_duration_s":0.002792,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001153,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001639,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026875,"completed_at":"2026-09-06T16:35:23.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 276.915541] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:23.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"824333d67ebde88eb20e9d45","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-824333d67ebde88eb20e9d45: start","job_status":"start","queue_duration_s":1.654795,"scheduling_latency_s":1.654829}1367gitlab # [ 276.929644] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.005Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"824333d67ebde88eb20e9d45","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-824333d67ebde88eb20e9d45: done: 0.025508 sec","job_status":"done","queue_duration_s":1.654795,"scheduling_latency_s":1.654829,"redis_calls":13,"redis_duration_s":0.0026420000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001618,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025508,"completed_at":"2026-09-06T16:35:24.005Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 276.969214] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.007Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37378f79c525ea54a9f70efd","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37378f79c525ea54a9f70efd: start","job_status":"start","queue_duration_s":1.682395,"scheduling_latency_s":1.682433}1369gitlab # [ 276.981573] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.034Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37378f79c525ea54a9f70efd","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37378f79c525ea54a9f70efd: done: 0.026357 sec","job_status":"done","queue_duration_s":1.682395,"scheduling_latency_s":1.682433,"redis_calls":13,"redis_duration_s":0.002776,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001029,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001747,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026357,"completed_at":"2026-09-06T16:35:24.033Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 277.021409] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80de30b5594ceee8752d85dc","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80de30b5594ceee8752d85dc: start","job_status":"start","queue_duration_s":1.711018,"scheduling_latency_s":1.711057}1371gitlab # [ 277.034088] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"80de30b5594ceee8752d85dc","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-80de30b5594ceee8752d85dc: done: 0.026659 sec","job_status":"done","queue_duration_s":1.711018,"scheduling_latency_s":1.711057,"redis_calls":13,"redis_duration_s":0.0030679999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00104,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002028,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026659,"completed_at":"2026-09-06T16:35:24.062Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 277.074171] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0c2cf4e7233efa9f4a6b989","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0c2cf4e7233efa9f4a6b989: start","job_status":"start","queue_duration_s":1.739795,"scheduling_latency_s":1.739831}1373gitlab # [ 277.086940] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0c2cf4e7233efa9f4a6b989","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0c2cf4e7233efa9f4a6b989: done: 0.025552 sec","job_status":"done","queue_duration_s":1.739795,"scheduling_latency_s":1.739831,"redis_calls":13,"redis_duration_s":0.00265,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001641,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016871,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025552,"completed_at":"2026-09-06T16:35:24.090Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 277.127284] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3793ff9cae653c52411281bb","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3793ff9cae653c52411281bb: start","job_status":"start","queue_duration_s":1.76738,"scheduling_latency_s":1.767418}1375gitlab # [ 277.140117] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3793ff9cae653c52411281bb","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3793ff9cae653c52411281bb: done: 0.02823 sec","job_status":"done","queue_duration_s":1.76738,"scheduling_latency_s":1.767418,"redis_calls":13,"redis_duration_s":0.002994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001885,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02823,"completed_at":"2026-09-06T16:35:24.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1376gitlab # [ 277.178963] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4503459c1a723c518a5a129f","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4503459c1a723c518a5a129f: start","job_status":"start","queue_duration_s":1.797825,"scheduling_latency_s":1.797863}1377gitlab # [ 277.191700] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4503459c1a723c518a5a129f","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4503459c1a723c518a5a129f: done: 0.02488 sec","job_status":"done","queue_duration_s":1.797825,"scheduling_latency_s":1.797863,"redis_calls":13,"redis_duration_s":0.00253,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000997,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001533,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02488,"completed_at":"2026-09-06T16:35:24.147Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1378gitlab # [ 277.231591] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f65fca5704feb2e99007abc9","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f65fca5704feb2e99007abc9: start","job_status":"start","queue_duration_s":1.82453,"scheduling_latency_s":1.824557}1379gitlab # [ 277.244449] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f65fca5704feb2e99007abc9","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f65fca5704feb2e99007abc9: done: 0.017626 sec","job_status":"done","queue_duration_s":1.82453,"scheduling_latency_s":1.824557,"redis_calls":13,"redis_duration_s":0.001901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001274,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017626,"completed_at":"2026-09-06T16:35:24.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 277.282971] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"870057395a09f77113972653","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-870057395a09f77113972653: start","job_status":"start","queue_duration_s":1.843558,"scheduling_latency_s":1.843584}1381gitlab # [ 277.295701] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"870057395a09f77113972653","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-870057395a09f77113972653: done: 0.018146 sec","job_status":"done","queue_duration_s":1.843558,"scheduling_latency_s":1.843584,"redis_calls":13,"redis_duration_s":0.001846,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018146,"completed_at":"2026-09-06T16:35:24.186Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1382gitlab # [ 277.335726] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"112378caf00a3d7c2698e187","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-112378caf00a3d7c2698e187: start","job_status":"start","queue_duration_s":1.863189,"scheduling_latency_s":1.863265}1383gitlab # [ 277.348780] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.207Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"112378caf00a3d7c2698e187","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-112378caf00a3d7c2698e187: done: 0.018524 sec","job_status":"done","queue_duration_s":1.863189,"scheduling_latency_s":1.863265,"redis_calls":13,"redis_duration_s":0.0018780000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001185,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018524,"completed_at":"2026-09-06T16:35:24.206Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1384gitlab # [ 277.389051] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.208Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874e0fb3ca52daefeafae54e","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874e0fb3ca52daefeafae54e: start","job_status":"start","queue_duration_s":1.883261,"scheduling_latency_s":1.883292}1385gitlab # [ 277.401348] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:35:24.227Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"874e0fb3ca52daefeafae54e","created_at":"2026-09-06T16:35:22.215Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-eb89a407642d4abb","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"58ba0eab85829ebd83d1f23138408d85","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:35:22.325Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-874e0fb3ca52daefeafae54e: done: 0.01871 sec","job_status":"done","queue_duration_s":1.883261,"scheduling_latency_s":1.883292,"redis_calls":13,"redis_duration_s":0.0018800000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000692,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01871,"completed_at":"2026-09-06T16:35:24.227Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 308.990399] setup-gitlab-start[3138]: W, [2026-09-06T16:35:56.933242 #3138] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1387gitlab # [ 310.066680] redis[792]: 10 changes in 300 seconds. Saving...1388gitlab # [ 310.067261] redis[792]: Background saving started by pid 33731389gitlab # [ 310.076877] redis[3373]: BGSAVE done, 420 keys saved, 0 keys skipped, 112487 bytes written.1390gitlab # [ 310.087773] redis[3373]: DB saved on disk1391gitlab # [ 310.090353] redis[3373]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1392gitlab # [ 310.168408] redis[792]: Background saving terminated with success1393gitlab # [ 311.768175] postgres[866]: [866] LOG: checkpoint starting: time1394gitlab # [ 312.717593] gitaly[893]: time="2026-09-06T16:36:00.659Z" level=info msg="burden monitor stats" pid=8931395gitlab # [ 313.672408] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:01.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2d0127523dd088bb2c3c6aed","created_at":"2026-09-06T16:36:01.585Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea53209fca087f2c7b2382e037c68871","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.609Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2d0127523dd088bb2c3c6aed: start","job_status":"start","queue_duration_s":0.004364,"scheduling_latency_s":0.004436}1396gitlab # [ 313.825973] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:01.769Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1397gitlab # [ 313.895431] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:01.838Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"2d0127523dd088bb2c3c6aed","created_at":"2026-09-06T16:36:01.585Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea53209fca087f2c7b2382e037c68871","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.609Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-2d0127523dd088bb2c3c6aed: done: 0.224157 sec","job_status":"done","queue_duration_s":0.004364,"scheduling_latency_s":0.004436,"redis_calls":15,"redis_duration_s":0.060249,"redis_read_bytes":905,"redis_write_bytes":1289,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.037901,"redis_feature_flag_read_bytes":900,"redis_feature_flag_write_bytes":463,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.017944,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004404,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.049669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.224157,"completed_at":"2026-09-06T16:36:01.838Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 313.949097] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:01.883Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e55e41954fc6cf332894428e","created_at":"2026-09-06T16:36:01.675Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8268459ef7ebd62ca247142998ea85e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.710Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e55e41954fc6cf332894428e: start","job_status":"start","queue_duration_s":0.172749,"scheduling_latency_s":0.172819}1399gitlab # [ 314.205457] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"e55e41954fc6cf332894428e","created_at":"2026-09-06T16:36:01.675Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8268459ef7ebd62ca247142998ea85e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.710Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-e55e41954fc6cf332894428e: done: 0.264893 sec","job_status":"done","queue_duration_s":0.172749,"scheduling_latency_s":0.172819,"redis_calls":13,"redis_duration_s":0.029523,"redis_read_bytes":139,"redis_write_bytes":1611,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.007773,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":386,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008633,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.044,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00211908,"exclusive_lock_hold_duration_s":0.04066146,"duration_s":0.264893,"completed_at":"2026-09-06T16:36:02.147Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047423,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 314.286820] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ee36ec1faa3835dac42c9b17","created_at":"2026-09-06T16:36:01.749Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68617344274785b6b468b439a42199c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:36:01.764Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ee36ec1faa3835dac42c9b17: start","job_status":"start","queue_duration_s":0.390792,"scheduling_latency_s":0.390862}1401gitlab # [ 314.369998] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.312Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"ee36ec1faa3835dac42c9b17","created_at":"2026-09-06T16:36:01.749Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"68617344274785b6b468b439a42199c8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:36:01.764Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-ee36ec1faa3835dac42c9b17: done: 0.157439 sec","job_status":"done","queue_duration_s":0.390792,"scheduling_latency_s":0.390862,"redis_calls":8,"redis_duration_s":0.024026,"redis_read_bytes":133,"redis_write_bytes":811,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.002302,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013944,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00778,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.157439,"completed_at":"2026-09-06T16:36:02.312Z","load_balancing_strategy":"primary","db_duration_s":0.010494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 314.446542] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ae663ff0d3c8c37ebaf984ab","created_at":"2026-09-06T16:36:01.781Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.797Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ae663ff0d3c8c37ebaf984ab: start","job_status":"start","queue_duration_s":0.537339,"scheduling_latency_s":0.53741}1403gitlab # [ 314.622606] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.565Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"ae663ff0d3c8c37ebaf984ab","created_at":"2026-09-06T16:36:01.781Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.797Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-ae663ff0d3c8c37ebaf984ab: done: 0.230472 sec","job_status":"done","queue_duration_s":0.537339,"scheduling_latency_s":0.53741,"redis_calls":14,"redis_duration_s":0.05925299999999999,"redis_read_bytes":16,"redis_write_bytes":4026,"redis_queues_calls":2,"redis_queues_duration_s":0.002415,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.036437,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.020401,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.230472,"completed_at":"2026-09-06T16:36:02.565Z","load_balancing_strategy":"primary","db_duration_s":0.001546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 314.711668] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.577Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d9d12434b25545c14a8cf476","created_at":"2026-09-06T16:36:01.822Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c98047ce65e8e7f3a7d7fa43f45e28a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:36:01.830Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d9d12434b25545c14a8cf476: start","job_status":"start","queue_duration_s":0.746967,"scheduling_latency_s":0.747035}1405gitlab # [ 314.734000] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.616Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1406gitlab # [ 314.739881] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"d9d12434b25545c14a8cf476","created_at":"2026-09-06T16:36:01.822Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0c98047ce65e8e7f3a7d7fa43f45e28a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:36:01.830Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-d9d12434b25545c14a8cf476: done: 0.080602 sec","job_status":"done","queue_duration_s":0.746967,"scheduling_latency_s":0.747035,"redis_calls":7,"redis_duration_s":0.003473,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002513,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080602,"completed_at":"2026-09-06T16:36:02.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1407gitlab # [ 314.788224] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"e726ca833c6f5c2f48329322","created_at":"2026-09-06T16:36:01.869Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c25c9027e24823d1db4184d3e336d597","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.893Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-e726ca833c6f5c2f48329322: start","job_status":"start","queue_duration_s":0.769119,"scheduling_latency_s":0.769181}1408gitlab # [ 314.892716] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Runners::StaleMachinesCleanupCronWorker","jid":"e726ca833c6f5c2f48329322","created_at":"2026-09-06T16:36:01.869Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c25c9027e24823d1db4184d3e336d597","meta.root_caller_id":"Cronjob","meta.feature_category":"fleet_visibility","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b74a831dd4b14c3fc2b303407a0e209ea633ae1f03f57327e51b9e7be5871f1f","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.893Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Runners::StaleMachinesCleanupCronWorker JID-e726ca833c6f5c2f48329322: done: 0.172906 sec","job_status":"done","queue_duration_s":0.769119,"scheduling_latency_s":0.769181,"redis_calls":10,"redis_duration_s":0.015113000000000001,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000872,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.013147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00477362,"extra.ci_runners_stale_machines_cleanup_cron_worker.status":"success","extra.ci_runners_stale_machines_cleanup_cron_worker.total_deleted":0,"extra.ci_runners_stale_machines_cleanup_cron_worker.batch_counts":[0],"duration_s":0.172906,"completed_at":"2026-09-06T16:36:02.835Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.069159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1409gitlab # [ 315.001413] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.853Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8f6b7784eb50a19e6123302e","created_at":"2026-09-06T16:36:01.905Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"177b351187e232f424542703bcba1d86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.913Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8f6b7784eb50a19e6123302e: start","job_status":"start","queue_duration_s":0.939279,"scheduling_latency_s":0.939347}1410gitlab # [ 315.024510] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.967Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8f6b7784eb50a19e6123302e","created_at":"2026-09-06T16:36:01.905Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"177b351187e232f424542703bcba1d86","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.913Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8f6b7784eb50a19e6123302e: done: 0.1147 sec","job_status":"done","queue_duration_s":0.939279,"scheduling_latency_s":0.939347,"redis_calls":7,"redis_duration_s":0.005092,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003389,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001703,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.1147,"completed_at":"2026-09-06T16:36:02.967Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 315.079191] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:02.972Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"79c0817cdbffbd30108a4c45","created_at":"2026-09-06T16:36:01.923Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"799c6beb069750fd8c384e107dcb2782","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.933Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-79c0817cdbffbd30108a4c45: start","job_status":"start","queue_duration_s":1.039036,"scheduling_latency_s":1.039098}1412gitlab # [ 315.094847] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.027Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"79c0817cdbffbd30108a4c45","created_at":"2026-09-06T16:36:01.923Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"799c6beb069750fd8c384e107dcb2782","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.933Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-79c0817cdbffbd30108a4c45: done: 0.05556 sec","job_status":"done","queue_duration_s":1.039036,"scheduling_latency_s":1.039098,"redis_calls":7,"redis_duration_s":0.0036220000000000002,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002465,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.05556,"completed_at":"2026-09-06T16:36:03.027Z","load_balancing_strategy":"primary","db_duration_s":0.009202,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1413gitlab # [ 315.140104] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a700ad6f203c6bedabcfe962","created_at":"2026-09-06T16:36:01.944Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1645bb7cf070748c7bdecb38c93d3cd3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.954Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a700ad6f203c6bedabcfe962: start","job_status":"start","queue_duration_s":1.076397,"scheduling_latency_s":1.076448}1414gitlab # [ 315.156003] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.051Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1415gitlab # [ 315.159872] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.065Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a700ad6f203c6bedabcfe962","created_at":"2026-09-06T16:36:01.944Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1645bb7cf070748c7bdecb38c93d3cd3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:01.954Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a700ad6f203c6bedabcfe962: done: 0.03432 sec","job_status":"done","queue_duration_s":1.076397,"scheduling_latency_s":1.076448,"redis_calls":7,"redis_duration_s":0.002928,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000856,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03432,"completed_at":"2026-09-06T16:36:03.065Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 315.203649] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.068Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"36d9bdeea00d3009de6c557c","created_at":"2026-09-06T16:36:02.006Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b04115b56a30e4434624fe98343b3ee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.011Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-36d9bdeea00d3009de6c557c: start","job_status":"start","queue_duration_s":1.056863,"scheduling_latency_s":1.056905}1417gitlab # [ 315.218357] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"36d9bdeea00d3009de6c557c","created_at":"2026-09-06T16:36:02.006Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0b04115b56a30e4434624fe98343b3ee","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.011Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-36d9bdeea00d3009de6c557c: done: 0.033267 sec","job_status":"done","queue_duration_s":1.056863,"scheduling_latency_s":1.056905,"redis_calls":9,"redis_duration_s":0.0027619999999999997,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001423,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00098182,"exclusive_lock_hold_duration_s":0.00640938,"duration_s":0.033267,"completed_at":"2026-09-06T16:36:03.101Z","load_balancing_strategy":"primary","db_duration_s":0.001745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1418gitlab # [ 315.259065] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.103Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f570f2829a630e7f6bc7464c","created_at":"2026-09-06T16:36:02.060Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1249af7a49e12cb810e800b804de77f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.081Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f570f2829a630e7f6bc7464c: start","job_status":"start","queue_duration_s":1.021753,"scheduling_latency_s":1.021793}1419gitlab # [ 315.275102] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.127Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"f570f2829a630e7f6bc7464c","created_at":"2026-09-06T16:36:02.060Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1249af7a49e12cb810e800b804de77f7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.081Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-f570f2829a630e7f6bc7464c: done: 0.024273 sec","job_status":"done","queue_duration_s":1.021753,"scheduling_latency_s":1.021793,"redis_calls":7,"redis_duration_s":0.001915,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024273,"completed_at":"2026-09-06T16:36:03.127Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1420gitlab # [ 315.319714] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.129Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"54aa128430c9f3ab5778d272","created_at":"2026-09-06T16:36:02.105Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"8928bcddcded6bd95646e3517da43de2","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.141Z","job_size_bytes":3,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-54aa128430c9f3ab5778d272: start","job_status":"start","queue_duration_s":0.988559,"scheduling_latency_s":0.988599}1421gitlab # [ 315.334376] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.161Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"54aa128430c9f3ab5778d272","created_at":"2026-09-06T16:36:02.105Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"8928bcddcded6bd95646e3517da43de2","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.141Z","job_size_bytes":3,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-54aa128430c9f3ab5778d272: done: 0.032161 sec","job_status":"done","queue_duration_s":0.988559,"scheduling_latency_s":0.988599,"redis_calls":10,"redis_duration_s":0.0024549999999999997,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00051,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001388,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000557,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019587,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004238,"duration_s":0.032161,"completed_at":"2026-09-06T16:36:03.161Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001134,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1422gitlab # [ 315.381059] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.164Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b14f8e02d2fed0181f7f8a8f","created_at":"2026-09-06T16:36:02.167Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4fc6e5db83e38fb071066643e900b60","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.193Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b14f8e02d2fed0181f7f8a8f: start","job_status":"start","queue_duration_s":0.970406,"scheduling_latency_s":0.970444}1423gitlab # [ 315.396528] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.202Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b14f8e02d2fed0181f7f8a8f","created_at":"2026-09-06T16:36:02.167Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e4fc6e5db83e38fb071066643e900b60","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.193Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b14f8e02d2fed0181f7f8a8f: done: 0.038096 sec","job_status":"done","queue_duration_s":0.970406,"scheduling_latency_s":0.970444,"redis_calls":10,"redis_duration_s":0.002842,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00917962,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.038096,"completed_at":"2026-09-06T16:36:03.202Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004073,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1424gitlab # [ 315.442336] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a68e4120ec7379219ea253db","created_at":"2026-09-06T16:36:02.243Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fec2e5e04e6acf66f1a3020fd660b64f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.259Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a68e4120ec7379219ea253db: start","job_status":"start","queue_duration_s":0.945164,"scheduling_latency_s":0.945201}1425gitlab # [ 315.457187] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.217Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1426gitlab # [ 315.460750] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.226Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"a68e4120ec7379219ea253db","created_at":"2026-09-06T16:36:02.243Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fec2e5e04e6acf66f1a3020fd660b64f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.259Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-a68e4120ec7379219ea253db: done: 0.022436 sec","job_status":"done","queue_duration_s":0.945164,"scheduling_latency_s":0.945201,"redis_calls":7,"redis_duration_s":0.0018750000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001293,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000582,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022436,"completed_at":"2026-09-06T16:36:03.226Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1427gitlab # [ 315.503698] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.228Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ef0efff2e8c0f7b43f525316","created_at":"2026-09-06T16:36:02.300Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8e98d9b1cf485838a331d9e8ba5f9b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.307Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ef0efff2e8c0f7b43f525316: start","job_status":"start","queue_duration_s":0.920896,"scheduling_latency_s":0.920933}1428gitlab # [ 315.518940] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.292Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"ef0efff2e8c0f7b43f525316","created_at":"2026-09-06T16:36:02.300Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8e98d9b1cf485838a331d9e8ba5f9b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.307Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-ef0efff2e8c0f7b43f525316: done: 0.0637 sec","job_status":"done","queue_duration_s":0.920896,"scheduling_latency_s":0.920933,"redis_calls":9,"redis_duration_s":0.002576,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001306,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00127,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035171,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00091428,"exclusive_lock_hold_duration_s":0.03822544,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.0637,"completed_at":"2026-09-06T16:36:03.292Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02133,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1429gitlab # [ 315.564408] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"42d268aae1771b9dde4db1ad","created_at":"2026-09-06T16:36:02.337Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0380a5da4a6e8d82729f2e13d562650d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.347Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-42d268aae1771b9dde4db1ad: start","job_status":"start","queue_duration_s":0.947441,"scheduling_latency_s":0.947478}1430gitlab # [ 315.579551] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.307Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1431gitlab # [ 315.585086] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.317Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"42d268aae1771b9dde4db1ad","created_at":"2026-09-06T16:36:02.337Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0380a5da4a6e8d82729f2e13d562650d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.347Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-42d268aae1771b9dde4db1ad: done: 0.022534 sec","job_status":"done","queue_duration_s":0.947441,"scheduling_latency_s":0.947478,"redis_calls":7,"redis_duration_s":0.001875,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00134,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000535,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022534,"completed_at":"2026-09-06T16:36:03.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 315.629927] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9da4c9a8192142c8b25d83de","created_at":"2026-09-06T16:36:02.375Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b376b3abc015b4879858f9aac5fc774c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:36:02.381Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9da4c9a8192142c8b25d83de: start","job_status":"start","queue_duration_s":0.937761,"scheduling_latency_s":0.937808}1433gitlab # [ 315.645068] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.332Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1434gitlab # [ 315.648548] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9da4c9a8192142c8b25d83de","created_at":"2026-09-06T16:36:02.375Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b376b3abc015b4879858f9aac5fc774c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:36:02.381Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9da4c9a8192142c8b25d83de: done: 0.022508 sec","job_status":"done","queue_duration_s":0.937761,"scheduling_latency_s":0.937808,"redis_calls":7,"redis_duration_s":0.001785,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001274,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022508,"completed_at":"2026-09-06T16:36:03.341Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 315.690998] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.343Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"73a567384d34595c539218d1","created_at":"2026-09-06T16:36:02.399Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b83d0f99e8507be897ee6a1c58bffd9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.426Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-73a567384d34595c539218d1: start","job_status":"start","queue_duration_s":0.91779,"scheduling_latency_s":0.917824}1436gitlab # [ 315.705772] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","jid":"73a567384d34595c539218d1","created_at":"2026-09-06T16:36:02.399Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b83d0f99e8507be897ee6a1c58bffd9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:805014656d48c4962f83ad00758ac8cfab4de244cfbd1d771454e66bd9018911","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.426Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker JID-73a567384d34595c539218d1: done: 0.059132 sec","job_status":"done","queue_duration_s":0.91779,"scheduling_latency_s":0.917824,"redis_calls":20,"redis_duration_s":0.005149,"redis_read_bytes":20,"redis_write_bytes":3247,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002278,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1346,"redis_queues_calls":2,"redis_queues_duration_s":0.00048,"redis_queues_read_bytes":3,"redis_queues_write_bytes":817,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001597,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":734,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000794,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":350,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040094,"duration_s":0.059132,"completed_at":"2026-09-06T16:36:03.402Z","load_balancing_strategy":"primary","db_duration_s":0.003757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 315.753568] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fabe2b7e1f8e4938720ca2ea","created_at":"2026-09-06T16:36:02.460Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a7e11b403447ccb7d6091262ae019ee9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.484Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fabe2b7e1f8e4938720ca2ea: start","job_status":"start","queue_duration_s":0.920771,"scheduling_latency_s":0.920806}1438gitlab # [ 315.769591] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"fabe2b7e1f8e4938720ca2ea","created_at":"2026-09-06T16:36:02.460Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a7e11b403447ccb7d6091262ae019ee9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.484Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-fabe2b7e1f8e4938720ca2ea: done: 0.029309 sec","job_status":"done","queue_duration_s":0.920771,"scheduling_latency_s":0.920806,"redis_calls":9,"redis_duration_s":0.0026230000000000003,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001322,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001301,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00087676,"exclusive_lock_hold_duration_s":0.00476208,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.029309,"completed_at":"2026-09-06T16:36:03.434Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 315.814456] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.436Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"21967f8e351bf8dde751cbe6","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-21967f8e351bf8dde751cbe6: start","job_status":"start","queue_duration_s":0.915786,"scheduling_latency_s":0.915819}1440gitlab # [ 315.827674] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.457Z","message":"No existing merge request to be cleaned up."}1441gitlab # [ 315.829693] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.474Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"21967f8e351bf8dde751cbe6","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-21967f8e351bf8dde751cbe6: done: 0.037954 sec","job_status":"done","queue_duration_s":0.915786,"scheduling_latency_s":0.915819,"redis_calls":10,"redis_duration_s":0.0032719999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002016,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037954,"completed_at":"2026-09-06T16:36:03.474Z","load_balancing_strategy":"primary","db_duration_s":0.004295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1442gitlab # [ 315.870078] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.476Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"01f14caa7536866cfe17df4f","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-01f14caa7536866cfe17df4f: start","job_status":"start","queue_duration_s":0.955801,"scheduling_latency_s":0.955839}1443gitlab # [ 315.883396] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.497Z","message":"No existing merge request to be cleaned up."}1444gitlab # [ 315.885150] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.512Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"01f14caa7536866cfe17df4f","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-01f14caa7536866cfe17df4f: done: 0.03656 sec","job_status":"done","queue_duration_s":0.955801,"scheduling_latency_s":0.955839,"redis_calls":10,"redis_duration_s":0.00276,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001163,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001597,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03656,"completed_at":"2026-09-06T16:36:03.512Z","load_balancing_strategy":"primary","db_duration_s":0.004651,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 315.924205] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.515Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8dd4be4f8859ff5e014a0b2","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8dd4be4f8859ff5e014a0b2: start","job_status":"start","queue_duration_s":0.994375,"scheduling_latency_s":0.994413}1446gitlab # [ 315.937265] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.534Z","message":"No existing merge request to be cleaned up."}1447gitlab # [ 315.939075] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.549Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"c8dd4be4f8859ff5e014a0b2","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-c8dd4be4f8859ff5e014a0b2: done: 0.034238 sec","job_status":"done","queue_duration_s":0.994375,"scheduling_latency_s":0.994413,"redis_calls":10,"redis_duration_s":0.002515,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001101,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001414,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034238,"completed_at":"2026-09-06T16:36:03.549Z","load_balancing_strategy":"primary","db_duration_s":0.003827,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1448gitlab # [ 315.980610] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.551Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8011d08be9de8afdb1ad46d4","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8011d08be9de8afdb1ad46d4: start","job_status":"start","queue_duration_s":1.030694,"scheduling_latency_s":1.03073}1449gitlab # [ 315.995431] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.571Z","message":"No existing merge request to be cleaned up."}1450gitlab # [ 315.997396] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.586Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8011d08be9de8afdb1ad46d4","created_at":"2026-09-06T16:36:02.457Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-d8f340e241a64b2b","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a360e7aa28f4148f13c813478523edb2","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.520Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8011d08be9de8afdb1ad46d4: done: 0.035616 sec","job_status":"done","queue_duration_s":1.030694,"scheduling_latency_s":1.03073,"redis_calls":10,"redis_duration_s":0.002674,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001138,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001536,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024332,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035616,"completed_at":"2026-09-06T16:36:03.586Z","load_balancing_strategy":"primary","db_duration_s":0.004102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 316.036654] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c4015450f2c8bb54ed6d1b5f","created_at":"2026-09-06T16:36:02.522Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"905f3486d1737ca25b25d2cd025e236e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.555Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c4015450f2c8bb54ed6d1b5f: start","job_status":"start","queue_duration_s":1.033201,"scheduling_latency_s":1.033238}1452gitlab # [ 316.052113] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"c4015450f2c8bb54ed6d1b5f","created_at":"2026-09-06T16:36:02.522Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"905f3486d1737ca25b25d2cd025e236e","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.555Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-c4015450f2c8bb54ed6d1b5f: done: 0.046944 sec","job_status":"done","queue_duration_s":1.033201,"scheduling_latency_s":1.033238,"redis_calls":10,"redis_duration_s":0.0026170000000000004,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000546,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001458,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000613,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042412,"duration_s":0.046944,"completed_at":"2026-09-06T16:36:03.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 316.098187] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:03.638Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"73483885ee92acac101ecf2a","created_at":"2026-09-06T16:36:02.580Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b970e898f2995a83ced997078c2897f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.590Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-73483885ee92acac101ecf2a: start","job_status":"start","queue_duration_s":1.047951,"scheduling_latency_s":1.047986}1454gitlab # [ 316.179997] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.123Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"73483885ee92acac101ecf2a","created_at":"2026-09-06T16:36:02.580Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b970e898f2995a83ced997078c2897f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.590Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-73483885ee92acac101ecf2a: done: 0.485709 sec","job_status":"done","queue_duration_s":1.047951,"scheduling_latency_s":1.047986,"redis_calls":562,"redis_duration_s":0.080718,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.079371,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.449954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.485709,"completed_at":"2026-09-06T16:36:04.123Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 316.237648] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.125Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c0bf9595dc7e20a8195d651e","created_at":"2026-09-06T16:36:02.644Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eab5b556a0c3c823d19646b0e0cc2a49","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.652Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c0bf9595dc7e20a8195d651e: start","job_status":"start","queue_duration_s":1.473256,"scheduling_latency_s":1.473292}1456gitlab # [ 316.253404] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.162Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c0bf9595dc7e20a8195d651e","created_at":"2026-09-06T16:36:02.644Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eab5b556a0c3c823d19646b0e0cc2a49","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.652Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c0bf9595dc7e20a8195d651e: done: 0.037064 sec","job_status":"done","queue_duration_s":1.473256,"scheduling_latency_s":1.473292,"redis_calls":7,"redis_duration_s":0.00196,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001418,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000542,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037064,"completed_at":"2026-09-06T16:36:04.162Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 316.295549] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"910616cb935ead2d870884bc","created_at":"2026-09-06T16:36:02.722Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d63b38d4db6ccfb7df6fca3285e6f100","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.731Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-910616cb935ead2d870884bc: start","job_status":"start","queue_duration_s":1.434049,"scheduling_latency_s":1.43409}1458gitlab # [ 316.597185] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.539Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"910616cb935ead2d870884bc","created_at":"2026-09-06T16:36:02.722Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d63b38d4db6ccfb7df6fca3285e6f100","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.731Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-910616cb935ead2d870884bc: done: 0.37414 sec","job_status":"done","queue_duration_s":1.434049,"scheduling_latency_s":1.43409,"redis_calls":11,"redis_duration_s":0.003601,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000581,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001547,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.295764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000916,"exclusive_lock_hold_duration_s":0.34436858,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.37414,"completed_at":"2026-09-06T16:36:04.539Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095969,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 316.694988] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.543Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"66abdb9604ea318845713257","created_at":"2026-09-06T16:36:02.793Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7502aa0e70282d0d4ab5cf6af8a442b9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.806Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-66abdb9604ea318845713257: start","job_status":"start","queue_duration_s":1.7368,"scheduling_latency_s":1.736833}1460gitlab # [ 316.713248] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"66abdb9604ea318845713257","created_at":"2026-09-06T16:36:02.793Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7502aa0e70282d0d4ab5cf6af8a442b9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.806Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-66abdb9604ea318845713257: done: 0.025214 sec","job_status":"done","queue_duration_s":1.7368,"scheduling_latency_s":1.736833,"redis_calls":7,"redis_duration_s":0.0024619999999999998,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0018,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000662,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025214,"completed_at":"2026-09-06T16:36:04.568Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 316.755877] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.570Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"1b1031204df92e670518cef4","created_at":"2026-09-06T16:36:02.838Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a966ee1f068754f7db076f79f8d8b2b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.863Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-1b1031204df92e670518cef4: start","job_status":"start","queue_duration_s":1.707575,"scheduling_latency_s":1.707606}1462gitlab # [ 316.771898] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.603Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"1b1031204df92e670518cef4","created_at":"2026-09-06T16:36:02.838Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a966ee1f068754f7db076f79f8d8b2b9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.863Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-1b1031204df92e670518cef4: done: 0.032328 sec","job_status":"done","queue_duration_s":1.707575,"scheduling_latency_s":1.707606,"redis_calls":12,"redis_duration_s":0.004256,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000814,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001856,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001586,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018627,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004409,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.032328,"completed_at":"2026-09-06T16:36:04.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 316.824023] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.606Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d9372768ec7a8386f2364c14","created_at":"2026-09-06T16:36:02.871Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.877Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d9372768ec7a8386f2364c14: start","job_status":"start","queue_duration_s":1.728816,"scheduling_latency_s":1.728848}1464gitlab # [ 316.839042] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d9372768ec7a8386f2364c14","created_at":"2026-09-06T16:36:02.871Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.877Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d9372768ec7a8386f2364c14: done: 0.151535 sec","job_status":"done","queue_duration_s":1.728816,"scheduling_latency_s":1.728848,"redis_calls":60,"redis_duration_s":0.021969,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.001448,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.019556,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.151535,"completed_at":"2026-09-06T16:36:04.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 316.884232] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.762Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"65e135b29640fe4024c729cb","created_at":"2026-09-06T16:36:02.885Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"649773e0d8b42f620c9d662a236504fc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.891Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-65e135b29640fe4024c729cb: start","job_status":"start","queue_duration_s":1.870306,"scheduling_latency_s":1.870368}1466gitlab # [ 316.910653] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.854Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"65e135b29640fe4024c729cb","created_at":"2026-09-06T16:36:02.885Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"649773e0d8b42f620c9d662a236504fc","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.891Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-65e135b29640fe4024c729cb: done: 0.091962 sec","job_status":"done","queue_duration_s":1.870306,"scheduling_latency_s":1.870368,"redis_calls":9,"redis_duration_s":0.004947999999999999,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002443,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.001659,"exclusive_lock_hold_duration_s":0.03532488,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.091962,"completed_at":"2026-09-06T16:36:04.853Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026067,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 316.958149] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.857Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7156202cfe24a5ccd69a598f","created_at":"2026-09-06T16:36:02.898Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"879b11d025e98445940a4c4ef9bdef48","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.911Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7156202cfe24a5ccd69a598f: start","job_status":"start","queue_duration_s":1.945446,"scheduling_latency_s":1.945491}1468gitlab # [ 316.971846] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.899Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"7156202cfe24a5ccd69a598f","created_at":"2026-09-06T16:36:02.898Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-87f47c68e9194d12","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"879b11d025e98445940a4c4ef9bdef48","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:36:02.911Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-7156202cfe24a5ccd69a598f: done: 0.042986 sec","job_status":"done","queue_duration_s":1.945446,"scheduling_latency_s":1.945491,"redis_calls":12,"redis_duration_s":0.005299,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000979,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000836,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002705,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000779,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055272,"duration_s":0.042986,"completed_at":"2026-09-06T16:36:04.899Z","load_balancing_strategy":"primary","db_duration_s":0.001066,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 317.016820] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"e5b2dc5c7ee522fb2e8a86dd","created_at":"2026-09-06T16:36:03.375Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-3006f69d88a245b5","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"4b83d0f99e8507be897ee6a1c58bffd9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:03.390Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-e5b2dc5c7ee522fb2e8a86dd: start","job_status":"start","queue_duration_s":1.511807,"scheduling_latency_s":1.511842}1470gitlab # [ 317.030050] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Projects::RefreshBuildArtifactsSizeStatisticsWorker","args":[],"jid":"e5b2dc5c7ee522fb2e8a86dd","created_at":"2026-09-06T16:36:03.375Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-3006f69d88a245b5","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Projects::ScheduleRefreshBuildArtifactsSizeStatisticsWorker","correlation_id":"4b83d0f99e8507be897ee6a1c58bffd9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:03.390Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::RefreshBuildArtifactsSizeStatisticsWorker JID-e5b2dc5c7ee522fb2e8a86dd: done: 0.065076 sec","job_status":"done","queue_duration_s":1.511807,"scheduling_latency_s":1.511842,"redis_calls":13,"redis_duration_s":0.005057,"redis_read_bytes":12,"redis_write_bytes":1665,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000829,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00177,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":519,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002458,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":801,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047948,"duration_s":0.065076,"completed_at":"2026-09-06T16:36:04.967Z","load_balancing_strategy":"primary","db_duration_s":0.009689,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 317.072492] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7ef86496ffdb1cb7fe9cb74","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7ef86496ffdb1cb7fe9cb74: start","job_status":"start","queue_duration_s":0.230541,"scheduling_latency_s":0.230576}1472gitlab # [ 317.086082] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:04.998Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7ef86496ffdb1cb7fe9cb74","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7ef86496ffdb1cb7fe9cb74: done: 0.02784 sec","job_status":"done","queue_duration_s":0.230541,"scheduling_latency_s":0.230576,"redis_calls":13,"redis_duration_s":0.003757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001275,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002482,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02784,"completed_at":"2026-09-06T16:36:04.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 317.125899] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68cbf4f5669c8c39da763f14","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68cbf4f5669c8c39da763f14: start","job_status":"start","queue_duration_s":0.260658,"scheduling_latency_s":0.26069}1474gitlab # [ 317.139435] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68cbf4f5669c8c39da763f14","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68cbf4f5669c8c39da763f14: done: 0.02686 sec","job_status":"done","queue_duration_s":0.260658,"scheduling_latency_s":0.26069,"redis_calls":13,"redis_duration_s":0.0036429999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001269,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002374,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017292,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02686,"completed_at":"2026-09-06T16:36:05.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 317.181165] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a70b03b469e77f31bd0f1414","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a70b03b469e77f31bd0f1414: start","job_status":"start","queue_duration_s":0.289779,"scheduling_latency_s":0.289817}1476gitlab # [ 317.193569] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a70b03b469e77f31bd0f1414","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a70b03b469e77f31bd0f1414: done: 0.039974 sec","job_status":"done","queue_duration_s":0.289779,"scheduling_latency_s":0.289817,"redis_calls":13,"redis_duration_s":0.007479,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039974,"completed_at":"2026-09-06T16:36:05.069Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 317.236021] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47e31f6ae3a568d7f59a4bf9","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47e31f6ae3a568d7f59a4bf9: start","job_status":"start","queue_duration_s":0.331952,"scheduling_latency_s":0.331988}1478gitlab # [ 317.249423] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47e31f6ae3a568d7f59a4bf9","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47e31f6ae3a568d7f59a4bf9: done: 0.02773 sec","job_status":"done","queue_duration_s":0.331952,"scheduling_latency_s":0.331988,"redis_calls":13,"redis_duration_s":0.0037819999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002531,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02773,"completed_at":"2026-09-06T16:36:05.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 317.290659] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1f9a149fa4d91f73f8c605f","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1f9a149fa4d91f73f8c605f: start","job_status":"start","queue_duration_s":0.362006,"scheduling_latency_s":0.362041}1480gitlab # [ 317.304287] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1f9a149fa4d91f73f8c605f","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1f9a149fa4d91f73f8c605f: done: 0.034107 sec","job_status":"done","queue_duration_s":0.362006,"scheduling_latency_s":0.362041,"redis_calls":13,"redis_duration_s":0.008361,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.006712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034107,"completed_at":"2026-09-06T16:36:05.135Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 317.344441] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa53e59d06880b18e605258e","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa53e59d06880b18e605258e: start","job_status":"start","queue_duration_s":0.398286,"scheduling_latency_s":0.39832}1482gitlab # [ 317.357902] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa53e59d06880b18e605258e","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa53e59d06880b18e605258e: done: 0.035226 sec","job_status":"done","queue_duration_s":0.398286,"scheduling_latency_s":0.39832,"redis_calls":13,"redis_duration_s":0.007889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019774,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035226,"completed_at":"2026-09-06T16:36:05.173Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 317.400484] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.175Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37124d4dc36e6dc5d4c007e5","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37124d4dc36e6dc5d4c007e5: start","job_status":"start","queue_duration_s":0.435808,"scheduling_latency_s":0.43584}1484gitlab # [ 317.413804] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37124d4dc36e6dc5d4c007e5","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37124d4dc36e6dc5d4c007e5: done: 0.027283 sec","job_status":"done","queue_duration_s":0.435808,"scheduling_latency_s":0.43584,"redis_calls":13,"redis_duration_s":0.0038399999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002558,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027283,"completed_at":"2026-09-06T16:36:05.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 317.458012] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1639c8602448f888d57293e","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1639c8602448f888d57293e: start","job_status":"start","queue_duration_s":0.465285,"scheduling_latency_s":0.465321}1486gitlab # [ 317.470905] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1639c8602448f888d57293e","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1639c8602448f888d57293e: done: 0.027365 sec","job_status":"done","queue_duration_s":0.465285,"scheduling_latency_s":0.465321,"redis_calls":13,"redis_duration_s":0.0038380000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002419,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027365,"completed_at":"2026-09-06T16:36:05.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 317.511835] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b24f244ec8961f4af231c08","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b24f244ec8961f4af231c08: start","job_status":"start","queue_duration_s":0.4949,"scheduling_latency_s":0.494933}1488gitlab # [ 317.525499] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b24f244ec8961f4af231c08","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b24f244ec8961f4af231c08: done: 0.026968 sec","job_status":"done","queue_duration_s":0.4949,"scheduling_latency_s":0.494933,"redis_calls":13,"redis_duration_s":0.003578,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002337,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026968,"completed_at":"2026-09-06T16:36:05.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 317.565204] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84100439bac7683b8e2e729","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84100439bac7683b8e2e729: start","job_status":"start","queue_duration_s":0.525173,"scheduling_latency_s":0.525209}1490gitlab # [ 317.578748] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.292Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f84100439bac7683b8e2e729","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f84100439bac7683b8e2e729: done: 0.027716 sec","job_status":"done","queue_duration_s":0.525173,"scheduling_latency_s":0.525209,"redis_calls":13,"redis_duration_s":0.003746,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001258,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002488,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027716,"completed_at":"2026-09-06T16:36:05.292Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 317.619374] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b5a8cdd0266308c05f1633b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b5a8cdd0266308c05f1633b: start","job_status":"start","queue_duration_s":0.555097,"scheduling_latency_s":0.555129}1492gitlab # [ 317.634609] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b5a8cdd0266308c05f1633b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b5a8cdd0266308c05f1633b: done: 0.03171 sec","job_status":"done","queue_duration_s":0.555097,"scheduling_latency_s":0.555129,"redis_calls":13,"redis_duration_s":0.003983,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002422,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.03171,"completed_at":"2026-09-06T16:36:05.326Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 317.674156] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.328Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e26491150b464c5916f03b40","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e26491150b464c5916f03b40: start","job_status":"start","queue_duration_s":0.589025,"scheduling_latency_s":0.589059}1494gitlab # [ 317.686864] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.355Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e26491150b464c5916f03b40","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e26491150b464c5916f03b40: done: 0.027229 sec","job_status":"done","queue_duration_s":0.589025,"scheduling_latency_s":0.589059,"redis_calls":13,"redis_duration_s":0.003628,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001215,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002413,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027229,"completed_at":"2026-09-06T16:36:05.355Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 317.739412] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dab85d74cedcacc19ef83f8c","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dab85d74cedcacc19ef83f8c: start","job_status":"start","queue_duration_s":0.619969,"scheduling_latency_s":0.620006}1496gitlab # [ 317.755985] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dab85d74cedcacc19ef83f8c","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dab85d74cedcacc19ef83f8c: done: 0.028214 sec","job_status":"done","queue_duration_s":0.619969,"scheduling_latency_s":0.620006,"redis_calls":13,"redis_duration_s":0.004568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003076,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028214,"completed_at":"2026-09-06T16:36:05.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 317.797964] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48ab0153c7f3e50ed1f7431d","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48ab0153c7f3e50ed1f7431d: start","job_status":"start","queue_duration_s":0.650793,"scheduling_latency_s":0.650828}1498gitlab # [ 317.812068] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48ab0153c7f3e50ed1f7431d","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48ab0153c7f3e50ed1f7431d: done: 0.028674 sec","job_status":"done","queue_duration_s":0.650793,"scheduling_latency_s":0.650828,"redis_calls":13,"redis_duration_s":0.004608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002599,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01829,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028674,"completed_at":"2026-09-06T16:36:05.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 317.856435] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b69f71ac63c12a0638183e0f","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b69f71ac63c12a0638183e0f: start","job_status":"start","queue_duration_s":0.681615,"scheduling_latency_s":0.681649}1500gitlab # [ 317.870249] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b69f71ac63c12a0638183e0f","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b69f71ac63c12a0638183e0f: done: 0.027085 sec","job_status":"done","queue_duration_s":0.681615,"scheduling_latency_s":0.681649,"redis_calls":13,"redis_duration_s":0.0037660000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001282,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002484,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017543,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027085,"completed_at":"2026-09-06T16:36:05.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 317.910907] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58f3a9f7aa7a881b97e307e6","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58f3a9f7aa7a881b97e307e6: start","job_status":"start","queue_duration_s":0.710877,"scheduling_latency_s":0.71091}1502gitlab # [ 317.924364] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.476Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58f3a9f7aa7a881b97e307e6","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58f3a9f7aa7a881b97e307e6: done: 0.026015 sec","job_status":"done","queue_duration_s":0.710877,"scheduling_latency_s":0.71091,"redis_calls":13,"redis_duration_s":0.003483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026015,"completed_at":"2026-09-06T16:36:05.476Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 317.966703] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cf6c680e9f24d57472e3fee","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cf6c680e9f24d57472e3fee: start","job_status":"start","queue_duration_s":0.740234,"scheduling_latency_s":0.740269}1504gitlab # [ 317.980662] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1cf6c680e9f24d57472e3fee","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1cf6c680e9f24d57472e3fee: done: 0.02668 sec","job_status":"done","queue_duration_s":0.740234,"scheduling_latency_s":0.740269,"redis_calls":13,"redis_duration_s":0.003726,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001368,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002358,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02668,"completed_at":"2026-09-06T16:36:05.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 318.023860] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.508Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0b78312bbceca33dabe378b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0b78312bbceca33dabe378b: start","job_status":"start","queue_duration_s":0.76915,"scheduling_latency_s":0.769185}1506gitlab # [ 318.039125] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0b78312bbceca33dabe378b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0b78312bbceca33dabe378b: done: 0.026245 sec","job_status":"done","queue_duration_s":0.76915,"scheduling_latency_s":0.769185,"redis_calls":13,"redis_duration_s":0.003745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001265,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00248,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01665,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026245,"completed_at":"2026-09-06T16:36:05.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 318.079051] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2dc1e6687fe0727d6cce576","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2dc1e6687fe0727d6cce576: start","job_status":"start","queue_duration_s":0.79777,"scheduling_latency_s":0.797806}1508gitlab # [ 318.093107] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2dc1e6687fe0727d6cce576","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2dc1e6687fe0727d6cce576: done: 0.026245 sec","job_status":"done","queue_duration_s":0.79777,"scheduling_latency_s":0.797806,"redis_calls":13,"redis_duration_s":0.00358,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001251,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002329,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016742,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026245,"completed_at":"2026-09-06T16:36:05.563Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 318.136238] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.565Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ede211851da0a0ef66d95f76","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ede211851da0a0ef66d95f76: start","job_status":"start","queue_duration_s":0.826202,"scheduling_latency_s":0.826238}1510gitlab # [ 318.149941] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ede211851da0a0ef66d95f76","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ede211851da0a0ef66d95f76: done: 0.027881 sec","job_status":"done","queue_duration_s":0.826202,"scheduling_latency_s":0.826238,"redis_calls":13,"redis_duration_s":0.004298,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001871,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002427,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027881,"completed_at":"2026-09-06T16:36:05.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 318.193741] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c758049f13297a5d0f441aea","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c758049f13297a5d0f441aea: start","job_status":"start","queue_duration_s":0.856303,"scheduling_latency_s":0.856334}1512gitlab # [ 318.207259] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c758049f13297a5d0f441aea","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c758049f13297a5d0f441aea: done: 0.026354 sec","job_status":"done","queue_duration_s":0.856303,"scheduling_latency_s":0.856334,"redis_calls":13,"redis_duration_s":0.0038650000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002435,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026354,"completed_at":"2026-09-06T16:36:05.622Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 318.256498] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83f013491abdef075b3297be","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83f013491abdef075b3297be: start","job_status":"start","queue_duration_s":0.88478,"scheduling_latency_s":0.88484}1514gitlab # [ 318.270279] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83f013491abdef075b3297be","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83f013491abdef075b3297be: done: 0.026235 sec","job_status":"done","queue_duration_s":0.88478,"scheduling_latency_s":0.88484,"redis_calls":13,"redis_duration_s":0.003875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001289,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002586,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026235,"completed_at":"2026-09-06T16:36:05.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 318.311387] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa1a2151b971e900dcca974f","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa1a2151b971e900dcca974f: start","job_status":"start","queue_duration_s":0.91327,"scheduling_latency_s":0.913302}1516gitlab # [ 318.325353] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fa1a2151b971e900dcca974f","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fa1a2151b971e900dcca974f: done: 0.026218 sec","job_status":"done","queue_duration_s":0.91327,"scheduling_latency_s":0.913302,"redis_calls":13,"redis_duration_s":0.0037270000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001332,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002395,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016632,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026218,"completed_at":"2026-09-06T16:36:05.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 318.368831] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc132b9bc62085fca5ec6e9d","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc132b9bc62085fca5ec6e9d: start","job_status":"start","queue_duration_s":0.941765,"scheduling_latency_s":0.941797}1518gitlab # [ 318.382570] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc132b9bc62085fca5ec6e9d","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc132b9bc62085fca5ec6e9d: done: 0.02547 sec","job_status":"done","queue_duration_s":0.941765,"scheduling_latency_s":0.941797,"redis_calls":13,"redis_duration_s":0.00343,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001214,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02547,"completed_at":"2026-09-06T16:36:05.706Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 318.425938] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.709Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828fb81fe8b7ab31077d2463","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828fb81fe8b7ab31077d2463: start","job_status":"start","queue_duration_s":0.970089,"scheduling_latency_s":0.970122}1520gitlab # [ 318.440463] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"828fb81fe8b7ab31077d2463","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-828fb81fe8b7ab31077d2463: done: 0.026351 sec","job_status":"done","queue_duration_s":0.970089,"scheduling_latency_s":0.970122,"redis_calls":13,"redis_duration_s":0.0036499999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001318,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002332,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016766,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026351,"completed_at":"2026-09-06T16:36:05.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 318.480934] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83feb09626fa93cea7903f48","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83feb09626fa93cea7903f48: start","job_status":"start","queue_duration_s":0.998697,"scheduling_latency_s":0.998726}1522gitlab # [ 318.493719] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83feb09626fa93cea7903f48","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83feb09626fa93cea7903f48: done: 0.028243 sec","job_status":"done","queue_duration_s":0.998697,"scheduling_latency_s":0.998726,"redis_calls":13,"redis_duration_s":0.0049510000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00167,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003281,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01749,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028243,"completed_at":"2026-09-06T16:36:05.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 318.533993] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc27e9d55a892ab9a2bb28e9","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc27e9d55a892ab9a2bb28e9: start","job_status":"start","queue_duration_s":1.029439,"scheduling_latency_s":1.029474}1524gitlab # [ 318.547000] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc27e9d55a892ab9a2bb28e9","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc27e9d55a892ab9a2bb28e9: done: 0.028046 sec","job_status":"done","queue_duration_s":1.029439,"scheduling_latency_s":1.029474,"redis_calls":13,"redis_duration_s":0.004088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002477,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017863,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028046,"completed_at":"2026-09-06T16:36:05.797Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 318.588541] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.799Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2eeada97cfa37db58a51ec17","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2eeada97cfa37db58a51ec17: start","job_status":"start","queue_duration_s":1.059736,"scheduling_latency_s":1.059767}1526gitlab # [ 318.601708] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.826Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2eeada97cfa37db58a51ec17","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2eeada97cfa37db58a51ec17: done: 0.02724 sec","job_status":"done","queue_duration_s":1.059736,"scheduling_latency_s":1.059767,"redis_calls":13,"redis_duration_s":0.0043230000000000005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002722,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02724,"completed_at":"2026-09-06T16:36:05.826Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 318.643922] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"638b08a088c43c2f05e0e10a","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-638b08a088c43c2f05e0e10a: start","job_status":"start","queue_duration_s":1.088843,"scheduling_latency_s":1.088867}1528gitlab # [ 318.657333] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.846Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"638b08a088c43c2f05e0e10a","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-638b08a088c43c2f05e0e10a: done: 0.018316 sec","job_status":"done","queue_duration_s":1.088843,"scheduling_latency_s":1.088867,"redis_calls":13,"redis_duration_s":0.003116,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001032,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018316,"completed_at":"2026-09-06T16:36:05.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 318.697780] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1a9c34592da7b2c1df5b007","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1a9c34592da7b2c1df5b007: start","job_status":"start","queue_duration_s":1.108776,"scheduling_latency_s":1.108812}1530gitlab # [ 318.710549] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b1a9c34592da7b2c1df5b007","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b1a9c34592da7b2c1df5b007: done: 0.018135 sec","job_status":"done","queue_duration_s":1.108776,"scheduling_latency_s":1.108812,"redis_calls":13,"redis_duration_s":0.003253,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001197,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002056,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018135,"completed_at":"2026-09-06T16:36:05.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 318.751858] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af9a35280b040758c04b4169","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af9a35280b040758c04b4169: start","job_status":"start","queue_duration_s":1.128646,"scheduling_latency_s":1.128666}1532gitlab # [ 318.765855] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"af9a35280b040758c04b4169","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-af9a35280b040758c04b4169: done: 0.019265 sec","job_status":"done","queue_duration_s":1.128646,"scheduling_latency_s":1.128666,"redis_calls":13,"redis_duration_s":0.003931,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001116,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019265,"completed_at":"2026-09-06T16:36:05.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 318.805899] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cd9698b6d115cdaa1897a73","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cd9698b6d115cdaa1897a73: start","job_status":"start","queue_duration_s":1.150173,"scheduling_latency_s":1.150193}1534gitlab # [ 318.819339] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5cd9698b6d115cdaa1897a73","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5cd9698b6d115cdaa1897a73: done: 0.019057 sec","job_status":"done","queue_duration_s":1.150173,"scheduling_latency_s":1.150193,"redis_calls":13,"redis_duration_s":0.003402,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001401,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019057,"completed_at":"2026-09-06T16:36:05.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 318.861438] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"476dff609b54b8a317a9b51b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-476dff609b54b8a317a9b51b: start","job_status":"start","queue_duration_s":1.171016,"scheduling_latency_s":1.171035}1536gitlab # [ 318.874525] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"476dff609b54b8a317a9b51b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-476dff609b54b8a317a9b51b: done: 0.019388 sec","job_status":"done","queue_duration_s":1.171016,"scheduling_latency_s":1.171035,"redis_calls":13,"redis_duration_s":0.003666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001223,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002443,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019388,"completed_at":"2026-09-06T16:36:05.930Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1537gitlab # [ 318.915991] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f1de50fb3f7728cadf5ae18","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f1de50fb3f7728cadf5ae18: start","job_status":"start","queue_duration_s":1.192168,"scheduling_latency_s":1.192192}1538gitlab # [ 318.928977] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.952Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f1de50fb3f7728cadf5ae18","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f1de50fb3f7728cadf5ae18: done: 0.020578 sec","job_status":"done","queue_duration_s":1.192168,"scheduling_latency_s":1.192192,"redis_calls":13,"redis_duration_s":0.0034999999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001237,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020578,"completed_at":"2026-09-06T16:36:05.952Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1539gitlab # [ 318.969099] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0419479666c0fe8a2caa99f9","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0419479666c0fe8a2caa99f9: start","job_status":"start","queue_duration_s":1.214606,"scheduling_latency_s":1.214629}1540gitlab # [ 318.982249] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0419479666c0fe8a2caa99f9","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0419479666c0fe8a2caa99f9: done: 0.018872 sec","job_status":"done","queue_duration_s":1.214606,"scheduling_latency_s":1.214629,"redis_calls":13,"redis_duration_s":0.0033489999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001135,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018872,"completed_at":"2026-09-06T16:36:05.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 319.023732] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3f3596dd72e8a130f46db3b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3f3596dd72e8a130f46db3b: start","job_status":"start","queue_duration_s":1.23566,"scheduling_latency_s":1.235679}1542gitlab # [ 319.037037] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c3f3596dd72e8a130f46db3b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c3f3596dd72e8a130f46db3b: done: 0.019484 sec","job_status":"done","queue_duration_s":1.23566,"scheduling_latency_s":1.235679,"redis_calls":13,"redis_duration_s":0.00398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001165,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019484,"completed_at":"2026-09-06T16:36:05.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1543gitlab # [ 319.079829] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:05.996Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d98dee5167228c254f5193c","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d98dee5167228c254f5193c: start","job_status":"start","queue_duration_s":1.256783,"scheduling_latency_s":1.256837}1544gitlab # [ 319.093240] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d98dee5167228c254f5193c","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d98dee5167228c254f5193c: done: 0.019003 sec","job_status":"done","queue_duration_s":1.256783,"scheduling_latency_s":1.256837,"redis_calls":13,"redis_duration_s":0.003554,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001254,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019003,"completed_at":"2026-09-06T16:36:06.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1545gitlab # [ 319.132616] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"185859ec4d8cbbd39a0d65b4","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-185859ec4d8cbbd39a0d65b4: start","job_status":"start","queue_duration_s":1.27758,"scheduling_latency_s":1.2776}1546gitlab # [ 319.145798] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"185859ec4d8cbbd39a0d65b4","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-185859ec4d8cbbd39a0d65b4: done: 0.019137 sec","job_status":"done","queue_duration_s":1.27758,"scheduling_latency_s":1.2776,"redis_calls":13,"redis_duration_s":0.0034089999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001136,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019137,"completed_at":"2026-09-06T16:36:06.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 319.187696] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"793f69c210157eed396d4b9e","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-793f69c210157eed396d4b9e: start","job_status":"start","queue_duration_s":1.29861,"scheduling_latency_s":1.298628}1548gitlab # [ 319.201064] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"793f69c210157eed396d4b9e","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-793f69c210157eed396d4b9e: done: 0.019193 sec","job_status":"done","queue_duration_s":1.29861,"scheduling_latency_s":1.298628,"redis_calls":13,"redis_duration_s":0.003136,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001097,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019193,"completed_at":"2026-09-06T16:36:06.057Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1549gitlab # [ 319.242388] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.059Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75815584c1b093ec69187b82","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75815584c1b093ec69187b82: start","job_status":"start","queue_duration_s":1.319495,"scheduling_latency_s":1.319513}1550gitlab # [ 319.255611] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"75815584c1b093ec69187b82","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-75815584c1b093ec69187b82: done: 0.018071 sec","job_status":"done","queue_duration_s":1.319495,"scheduling_latency_s":1.319513,"redis_calls":13,"redis_duration_s":0.003144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001093,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018071,"completed_at":"2026-09-06T16:36:06.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1551gitlab # [ 319.298411] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"154277362898ffc5a1d639c7","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-154277362898ffc5a1d639c7: start","job_status":"start","queue_duration_s":1.33923,"scheduling_latency_s":1.339248}1552gitlab # [ 319.311699] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"154277362898ffc5a1d639c7","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-154277362898ffc5a1d639c7: done: 0.019162 sec","job_status":"done","queue_duration_s":1.33923,"scheduling_latency_s":1.339248,"redis_calls":13,"redis_duration_s":0.003788,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001458,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011759,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019162,"completed_at":"2026-09-06T16:36:06.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1553gitlab # [ 319.354048] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1afdcc23ea015fdc3ee2aadc","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1afdcc23ea015fdc3ee2aadc: start","job_status":"start","queue_duration_s":1.360038,"scheduling_latency_s":1.360058}1554gitlab # [ 319.367117] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1afdcc23ea015fdc3ee2aadc","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1afdcc23ea015fdc3ee2aadc: done: 0.019322 sec","job_status":"done","queue_duration_s":1.360038,"scheduling_latency_s":1.360058,"redis_calls":13,"redis_duration_s":0.004088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001839,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019322,"completed_at":"2026-09-06T16:36:06.118Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1555gitlab # [ 319.407263] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99272d2293a51ab5834b3532","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99272d2293a51ab5834b3532: start","job_status":"start","queue_duration_s":1.381236,"scheduling_latency_s":1.381254}1556gitlab # [ 319.420063] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99272d2293a51ab5834b3532","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99272d2293a51ab5834b3532: done: 0.018407 sec","job_status":"done","queue_duration_s":1.381236,"scheduling_latency_s":1.381254,"redis_calls":13,"redis_duration_s":0.003311,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001075,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018407,"completed_at":"2026-09-06T16:36:06.139Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1557gitlab # [ 319.461851] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4352937ea501ca03754dd65","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4352937ea501ca03754dd65: start","job_status":"start","queue_duration_s":1.401341,"scheduling_latency_s":1.40136}1558gitlab # [ 319.474936] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.159Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4352937ea501ca03754dd65","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4352937ea501ca03754dd65: done: 0.018228 sec","job_status":"done","queue_duration_s":1.401341,"scheduling_latency_s":1.40136,"redis_calls":13,"redis_duration_s":0.00324,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002177,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018228,"completed_at":"2026-09-06T16:36:06.159Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1559gitlab # [ 319.514997] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b53265d40fb2e8315aa0d5df","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b53265d40fb2e8315aa0d5df: start","job_status":"start","queue_duration_s":1.421428,"scheduling_latency_s":1.421446}1560gitlab # [ 319.527893] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.179Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b53265d40fb2e8315aa0d5df","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b53265d40fb2e8315aa0d5df: done: 0.018701 sec","job_status":"done","queue_duration_s":1.421428,"scheduling_latency_s":1.421446,"redis_calls":13,"redis_duration_s":0.003778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002691,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018701,"completed_at":"2026-09-06T16:36:06.179Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1561gitlab # [ 319.568793] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c49d994c3a108507ad8e2c3","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c49d994c3a108507ad8e2c3: start","job_status":"start","queue_duration_s":1.441638,"scheduling_latency_s":1.441694}1562gitlab # [ 319.581669] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c49d994c3a108507ad8e2c3","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c49d994c3a108507ad8e2c3: done: 0.020378 sec","job_status":"done","queue_duration_s":1.441638,"scheduling_latency_s":1.441694,"redis_calls":13,"redis_duration_s":0.0024749999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001575,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020378,"completed_at":"2026-09-06T16:36:06.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1563gitlab # [ 319.623072] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d082d7761bc9b67e73b34093","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d082d7761bc9b67e73b34093: start","job_status":"start","queue_duration_s":1.46348,"scheduling_latency_s":1.4635}1564gitlab # [ 319.636148] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d082d7761bc9b67e73b34093","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d082d7761bc9b67e73b34093: done: 0.019047 sec","job_status":"done","queue_duration_s":1.46348,"scheduling_latency_s":1.4635,"redis_calls":13,"redis_duration_s":0.00231,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000836,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001474,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019047,"completed_at":"2026-09-06T16:36:06.222Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1565gitlab # [ 319.678575] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.223Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c2a73f262f2e14d85f1620b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c2a73f262f2e14d85f1620b: start","job_status":"start","queue_duration_s":1.483932,"scheduling_latency_s":1.483951}1566gitlab # [ 319.691696] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c2a73f262f2e14d85f1620b","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c2a73f262f2e14d85f1620b: done: 0.017092 sec","job_status":"done","queue_duration_s":1.483932,"scheduling_latency_s":1.483951,"redis_calls":13,"redis_duration_s":0.002039,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001315,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017092,"completed_at":"2026-09-06T16:36:06.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1567gitlab # [ 319.733292] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.242Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae3251bc042d43739543e356","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae3251bc042d43739543e356: start","job_status":"start","queue_duration_s":1.502336,"scheduling_latency_s":1.502355}1568gitlab # [ 319.746337] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae3251bc042d43739543e356","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae3251bc042d43739543e356: done: 0.016991 sec","job_status":"done","queue_duration_s":1.502336,"scheduling_latency_s":1.502355,"redis_calls":13,"redis_duration_s":0.002294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001567,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016991,"completed_at":"2026-09-06T16:36:06.258Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1569gitlab # [ 319.788217] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9321b8754e31f23b1dfdab4","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9321b8754e31f23b1dfdab4: start","job_status":"start","queue_duration_s":1.520649,"scheduling_latency_s":1.520668}1570gitlab # [ 319.801490] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:06.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9321b8754e31f23b1dfdab4","created_at":"2026-09-06T16:36:04.619Z","trace_propagation_headers":{"sentry-trace":"8abb21726c8140f3a8050d001432080a-33ee553b062d444c","baggage":"sentry-trace_id=8abb21726c8140f3a8050d001432080a,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"12660219be9468bc2a229e186f5c8e87","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:36:04.739Z","job_size_bytes":2,"pid":2115,"sidekiq_tid":"m7mz","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9321b8754e31f23b1dfdab4: done: 0.01877 sec","job_status":"done","queue_duration_s":1.520649,"scheduling_latency_s":1.520668,"redis_calls":13,"redis_duration_s":0.002804,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01877,"completed_at":"2026-09-06T16:36:06.279Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1571gitlab # [ 357.773494] postgres[3535]: [3535] ERROR: relation "table_without_models" does not exist1572gitlab # [ 357.777430] postgres[3535]: [3535] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1573gitlab # [ 364.546126] runuser[3137]: pam_unix(runuser:session): session closed for user gitlab1574gitlab # [ 364.548302] setup-gitlab-start[3083]: + systemctl restart gitlab-sidekiq.service1575gitlab # [ 364.595222] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:52.538Z","message":"Shutting down"}1576gitlab # [ 364.598832] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:52.540Z","message":"Scheduler exiting..."}1577gitlab # [ 364.600877] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:52.541Z","message":"Terminating quiet threads for default capsule"}1578gitlab # [ 364.603164] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:52.542Z","message":"Scheduler exiting..."}1579gitlab # [ 364.605595] systemd[1]: Stopping gitlab-sidekiq.service...1580gitlab # [ 365.100663] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:53.044Z","message":"Pausing to allow jobs to finish..."}1581gitlab # [ 369.111418] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:57.053Z","message":"Bye!"}1582gitlab # [ 369.163983] sidekiq-cluster[2115]: {"severity":"INFO","time":"2026-09-06T16:36:57.107Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2115,"worker_id":"sidekiq_0","memwd_rss_bytes":972136448,"retry":0}1583gitlab # [ 370.227919] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1584gitlab # [ 370.228479] systemd[1]: Stopped gitlab-sidekiq.service.1585gitlab # [ 370.228777] systemd[1]: gitlab-sidekiq.service: Consumed 2min 17.445s CPU time over 3min 55.009s wall clock time, 954.8M memory peak, 4.3M read from disk.1586gitlab # [ 370.242375] systemd[1]: Started gitlab-sidekiq.service.1587gitlab # [ 370.246651] setup-gitlab-start[3083]: + mkdir -p /var/lib/secrets1588gitlab # [ 370.252720] setup-gitlab-start[3083]: + echo nixbot-test-token-of-20-chars1589gitlab # [ 370.255654] setup-gitlab-start[3083]: + chmod 644 /var/lib/secrets/gitlab-token1590gitlab # [ 370.259304] setup-gitlab-start[3083]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1591gitlab # [ 370.777310] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:36:58 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 4661592gitlab # [ 373.311354] gitaly[893]: time="2026-09-06T16:37:01.255Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:11.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:01.254" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593gitlab # [ 373.311941] gitaly[893]: time="2026-09-06T16:37:01.256Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:11.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:01.254" grpc.time_ms=1.32 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1594gitlab # [ 374.596434] gitaly[893]: time="2026-09-06T16:37:02.539Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.288" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.539" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1595gitlab # [ 374.596949] gitaly[893]: time="2026-09-06T16:37:02.539Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.288" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.539" grpc.time_ms=0.586 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1596gitlab # [ 374.710844] gitaly[893]: time="2026-09-06T16:37:02.655Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA diskcache=3b69027f-5889-4d51-84bd-703b87e041db grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:37:12.298" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.549" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1597gitlab # [ 374.740690] gitaly[893]: time="2026-09-06T16:37:02.655Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=30 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=77 command.spawn_token_fork_ms=7 command.system_time_ms=30 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:37:12.298" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.549" grpc.time_ms=105.545 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1598gitlab # [ 374.768731] gitaly[893]: time="2026-09-06T16:37:02.683Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.682" pid=893 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1599gitlab # [ 374.784568] gitaly[893]: time="2026-09-06T16:37:02.684Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.682" grpc.time_ms=1.509 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1600gitlab # [ 374.800693] gitaly[893]: time="2026-09-06T16:37:02.712Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:37:12.013" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.694" grpc.time_ms=18.055 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1601gitlab # [ 374.908253] gitaly[893]: time="2026-09-06T16:37:02.852Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.171" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.851" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1602gitlab # [ 374.940123] gitaly[893]: time="2026-09-06T16:37:02.852Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.171" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.851" grpc.time_ms=0.649 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1603gitlab # [ 374.975275] gitaly[893]: time="2026-09-06T16:37:02.900Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA diskcache=60d689c4-2b75-4db1-8e76-62851551ca32 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:37:12.175" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.856" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1604gitlab # [ 374.998497] gitaly[893]: time="2026-09-06T16:37:02.901Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=12 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-06T16:37:12.175" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.856" grpc.time_ms=44.569 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1605gitlab # [ 375.028980] gitaly[893]: time="2026-09-06T16:37:02.916Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.233" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.914" pid=893 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1606gitlab # [ 375.045239] gitaly[893]: time="2026-09-06T16:37:02.916Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:12.233" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.914" grpc.time_ms=1.865 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1607gitlab # [ 375.059023] gitaly[893]: time="2026-09-06T16:37:02.936Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=5 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:37:12.243" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:02.924" grpc.time_ms=11.765 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1608gitlab # [ 375.076066] gitaly[893]: time="2026-09-06T16:37:03.005Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:37:13.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:03.004" grpc.time_ms=0.239 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1609gitlab # [ 375.089477] gitaly[893]: time="2026-09-06T16:37:03.019Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:37:13.018" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:03.018" grpc.time_ms=0.36 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1610gitlab # [ 375.794682] sidekiq-cluster[3611]: {"severity":"INFO","time":"2026-09-06T16:37:03.704Z","message":"Starting cluster with 1 processes"}1611gitlab # [ 375.879016] gitaly[893]: time="2026-09-06T16:37:03.822Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS92QMD40TK476DKRDFEBA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:37:13.123" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:03.803" grpc.time_ms=18.133 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1612gitlab # [ 376.301300] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:04 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 55031613gitlab # [ 376.307657] setup-gitlab-start[3624]: {"id":1,"message":null,"error":null}1614gitlab # [ 381.572499] gitaly[893]: time="2026-09-06T16:37:09.515Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9D126YV1XAFD9MPTARSW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:37:19.246" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:09.497" grpc.time_ms=18.106 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1615gitlab # [ 381.730933] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:09 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 3901616gitlab # [ 381.738198] setup-gitlab-start[3083]: + mkdir -p /tmp/test-flake1617gitlab # [ 381.741783] setup-gitlab-start[3083]: + cd /tmp/test-flake1618gitlab # [ 381.744977] setup-gitlab-start[3083]: + git init -b master1619gitlab # [ 381.772878] setup-gitlab-start[3716]: Initialized empty Git repository in /tmp/test-flake/.git/1620gitlab # [ 381.776609] setup-gitlab-start[3083]: + git config user.name test1621gitlab # [ 381.781029] setup-gitlab-start[3083]: + git config user.email test@example.com1622gitlab # [ 381.788716] setup-gitlab-start[3083]: + cat1623gitlab # [ 381.794944] setup-gitlab-start[3083]: + git add flake.nix1624gitlab # [ 381.804515] setup-gitlab-start[3083]: + git commit -m 'initial commit'1625gitlab # [ 381.837075] setup-gitlab-start[3721]: [master (root-commit) 8e23894] initial commit1626gitlab # [ 381.842422] setup-gitlab-start[3721]: 1 file changed, 10 insertions(+)1627gitlab # [ 381.845454] setup-gitlab-start[3721]: create mode 100644 flake.nix1628gitlab # [ 381.848666] setup-gitlab-start[3083]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1629gitlab # [ 381.854057] setup-gitlab-start[3083]: + git push -u origin master1630gitlab # [ 381.995936] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:09 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 931631gitlab # [ 382.506247] gitaly[893]: time="2026-09-06T16:37:10.450Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9DNNHT0D1KFKRQ6DSGBR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:10.438" grpc.time_ms=11.633 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1632gitlab # [ 382.520620] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:10 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 5101633gitlab # [ 382.883998] gitaly[893]: time="2026-09-06T16:37:10.825Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:37:20.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:10.808" grpc.time_ms=16.729 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1634gitlab # [ 382.982230] gitaly[893]: time="2026-09-06T16:37:10.926Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:37:40.006" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:37:10.906" grpc.time_ms=19.347 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1635gitlab # [ 383.111349] gitaly[893]: time="2026-09-06T16:37:11.055Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 duration_ms=379ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/allowed"1636gitlab # [ 383.203151] gitaly[893]: time="2026-09-06T16:37:11.147Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 duration_ms=90ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/pre_receive"1637gitlab # [ 383.212624] gitaly[893]: time="2026-09-06T16:37:11.148Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5315 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:37:10.673" grpc.time_ms=474.686 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1638gitlab # [ 383.270622] gitaly[893]: time="2026-09-06T16:37:11.214Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:37:11.212" grpc.time_ms=1.861 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1639gitlab # [ 383.332765] gitaly[893]: time="2026-09-06T16:37:11.277Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:37:11.275" grpc.time_ms=1.272 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1640gitlab # [ 383.392546] gitaly[893]: time="2026-09-06T16:37:11.336Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:37:11.335" grpc.time_ms=1.374 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1641gitlab # [ 383.608029] gitaly[893]: time="2026-09-06T16:37:11.552Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:37:21.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:11.534" grpc.time_ms=17.683 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1642gitlab # [ 383.667372] gitaly[893]: time="2026-09-06T16:37:11.611Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=0 command.real_time_ms=36 command.spawn_token_fork_ms=4 command.system_time_ms=20 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:37:21.038" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:11.569" grpc.time_ms=41.836 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1643gitlab # [ 383.695194] gitaly[893]: time="2026-09-06T16:37:11.639Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 duration_ms=239ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/post_receive"1644gitlab # [ 383.699116] gitaly[893]: time="2026-09-06T16:37:11.641Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4809 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:37:11.399" grpc.time_ms=241.33 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1645gitlab # [ 383.712534] gitaly[893]: time="2026-09-06T16:37:11.656Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 diskcache=e60d3816-8f57-4fcc-8229-ac9725a7c092 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:10.561" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1646gitlab # [ 383.724469] gitaly[893]: time="2026-09-06T16:37:11.657Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=379 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13438 command.oublock=32 command.real_time_ms=1085 command.spawn_token_fork_ms=2 command.system_time_ms=228 command.user_time_ms=151 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:10.561" grpc.time_ms=1095.77 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1647gitlab # [ 383.740489] setup-gitlab-start[3725]: To http://gitlab/root/test-flake.git1648gitlab # [ 383.741512] setup-gitlab-start[3725]: * [new branch] master -> master1649gitlab # [ 383.742527] setup-gitlab-start[3725]: branch 'master' set up to track 'origin/master'.1650gitlab # [ 383.744106] gitlab-workhorse[823]: time="2026-09-06T16:37:11Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521651gitlab # [ 383.746123] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:11 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 11821652gitlab # [ 383.748463] systemd[1]: Finished setup-gitlab.service.1653gitlab # [ 383.749368] systemd[1]: Starting nixbot CI service...1654gitlab: (finished: waiting for unit setup-gitlab.service, in 385.12 seconds)1655gitlab: waiting for unit nixbot.service1656gitlab # [ 385.014829] nixbot[3813]: {"ts": "2026-09-06T16:37:12.958Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1657gitlab # [ 385.055870] nixbot[3813]: {"ts": "2026-09-06T16:37:12.999Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1658gitlab # [ 385.097956] nixbot[3813]: {"ts": "2026-09-06T16:37:13.042Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1659gitlab # [ 385.105209] nixbot[3813]: {"ts": "2026-09-06T16:37:13.049Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1660gitlab # [ 385.112057] nixbot[3813]: {"ts": "2026-09-06T16:37:13.055Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1661gitlab # [ 385.120365] nixbot[3813]: {"ts": "2026-09-06T16:37:13.064Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1662gitlab # [ 385.126849] nixbot[3813]: {"ts": "2026-09-06T16:37:13.070Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1663gitlab # [ 385.133634] nixbot[3813]: {"ts": "2026-09-06T16:37:13.073Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1664gitlab # [ 385.140269] nixbot[3813]: {"ts": "2026-09-06T16:37:13.082Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1665gitlab # [ 385.146887] nixbot[3813]: {"ts": "2026-09-06T16:37:13.086Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1666gitlab # [ 385.153866] nixbot[3813]: {"ts": "2026-09-06T16:37:13.093Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1667gitlab # [ 385.160382] nixbot[3813]: {"ts": "2026-09-06T16:37:13.103Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1668gitlab # [ 385.167023] nixbot[3813]: {"ts": "2026-09-06T16:37:13.108Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1669gitlab # [ 385.174003] nixbot[3813]: {"ts": "2026-09-06T16:37:13.111Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1670gitlab # [ 385.181661] nixbot[3813]: {"ts": "2026-09-06T16:37:13.116Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1671gitlab # [ 385.187958] nixbot[3813]: {"ts": "2026-09-06T16:37:13.119Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1672gitlab # [ 385.193591] nixbot[3813]: {"ts": "2026-09-06T16:37:13.122Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1673gitlab # [ 385.199323] nixbot[3813]: {"ts": "2026-09-06T16:37:13.124Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1674gitlab # [ 385.204778] nixbot[3813]: {"ts": "2026-09-06T16:37:13.133Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1675gitlab # [ 385.210949] nixbot[3813]: {"ts": "2026-09-06T16:37:13.136Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1676gitlab # [ 385.216461] nixbot[3813]: {"ts": "2026-09-06T16:37:13.138Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1677gitlab # [ 385.221625] nixbot[3813]: {"ts": "2026-09-06T16:37:13.148Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1678gitlab # [ 385.227007] nixbot[3813]: {"ts": "2026-09-06T16:37:13.151Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1679gitlab # [ 385.231992] nixbot[3813]: {"ts": "2026-09-06T16:37:13.153Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1680gitlab # [ 385.236200] nixbot[3813]: {"ts": "2026-09-06T16:37:13.156Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1681gitlab # [ 385.240527] nixbot[3813]: {"ts": "2026-09-06T16:37:13.160Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1682gitlab # [ 385.244771] nixbot[3813]: {"ts": "2026-09-06T16:37:13.163Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1683gitlab # [ 385.249221] nixbot[3813]: {"ts": "2026-09-06T16:37:13.165Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1684gitlab # [ 385.253652] nixbot[3813]: {"ts": "2026-09-06T16:37:13.167Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1685gitlab # [ 385.257961] nixbot[3813]: {"ts": "2026-09-06T16:37:13.186Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1686gitlab # [ 385.262048] nixbot[3813]: {"ts": "2026-09-06T16:37:13.194Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1687gitlab # [ 385.266039] nixbot[3813]: {"ts": "2026-09-06T16:37:13.197Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1688gitlab # [ 385.269829] nixbot[3813]: {"ts": "2026-09-06T16:37:13.199Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "public_badges"}1689gitlab # [ 385.764193] nixbot[3813]: INFO: Started server process [3813]1690gitlab # [ 385.767112] nixbot[3813]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1691gitlab # [ 385.833094] nixbot[3813]: INFO: ::ffff:127.0.0.1:58420 - "GET /health HTTP/1.1" 200 OK1692gitlab # [ 385.843736] systemd[1]: Started nixbot CI service.1693gitlab # [ 385.848302] systemd[1]: Reached target Multi-User System.1694gitlab # [ 385.851066] systemd[1]: Startup finished in 644ms (kernel) + 4.923s (initrd) + 6min 20.280s (userspace) = 6min 25.848s.1695gitlab: (finished: waiting for unit nixbot.service, in 2.16 seconds)1696gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1697gitlab # [ 386.860669] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:14 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 11221698gitlab # [ 386.863493] nixbot[3813]: {"ts": "2026-09-06T16:37:14.806Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1699gitlab # [ 386.890193] nixbot[3813]: INFO: ::ffff:127.0.0.1:58426 - "GET /health HTTP/1.1" 200 OK1700gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.06 seconds)1701subtest: project discovered and webhook registered1702gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1703gitlab # [ 386.905079] nixbot[3813]: {"ts": "2026-09-06T16:37:14.849Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1704gitlab # [ 387.040649] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:14 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1191705gitlab # [ 387.044380] nixbot[3813]: {"ts": "2026-09-06T16:37:14.985Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1706gitlab # [ 387.047087] nixbot[3813]: {"ts": "2026-09-06T16:37:14.988Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1707gitlab # [ 387.065124] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 1321708gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.18 seconds)1709gitlab # [ 387.248126] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1981710gitlab # [ 387.254908] nixbot[3813]: {"ts": "2026-09-06T16:37:15.193Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1711gitlab # [ 387.483582] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:15 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2241712gitlab # [ 387.491277] nixbot[3813]: {"ts": "2026-09-06T16:37:15.428Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1713gitlab # [ 387.686178] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:15 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 1871714gitlab # [ 387.688847] nixbot[3813]: {"ts": "2026-09-06T16:37:15.632Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1715gitlab # [ 387.851243] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:15 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 1531716gitlab # [ 387.859381] nixbot[3813]: {"ts": "2026-09-06T16:37:15.797Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1717gitlab # [ 388.018681] gitaly[893]: time="2026-09-06T16:37:15.961Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9KD1XAGY6BKMTSBD1BBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-06T16:37:45.033" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:15.933" grpc.time_ms=27.133 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1718gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1719gitlab # [ 388.059527] gitaly[893]: time="2026-09-06T16:37:15.994Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9KD1XAGY6BKMTSBD1BBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:37:45.092" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:37:15.993" grpc.time_ms=1.421 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1720gitlab # [ 388.086063] gitaly[893]: time="2026-09-06T16:37:16.016Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9KD1XAGY6BKMTSBD1BBZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-06T16:37:26.000" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:37:16.001" grpc.time_ms=14.353 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1721gitlab # [ 388.132897] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2661722gitlab # [ 388.137148] nixbot[3813]: {"ts": "2026-09-06T16:37:16.080Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1723gitlab # [ 388.264149] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 1201724gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.20 seconds)1725(finished: subtest: project discovered and webhook registered, in 1.37 seconds)1726subtest: push delivers the webhook1727gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1728gitlab # [ 388.301474] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:16 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1541729gitlab # [ 388.305477] nixbot[3813]: {"ts": "2026-09-06T16:37:16.249Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1730gitlab # [ 388.310015] nixbot[3813]: {"ts": "2026-09-06T16:37:16.254Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1731gitlab # [ 388.460447] gitaly[893]: time="2026-09-06T16:37:16.404Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9KWKPNXB5EG41QYDM12X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:16.388" grpc.time_ms=15.951 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1732gitlab # [ 388.501605] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 971733gitlab # [ 388.695280] gitaly[893]: time="2026-09-06T16:37:16.639Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9KZQPER5834EDRFG5BSH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:16.618" pid=893 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1734gitlab # [ 388.728788] gitaly[893]: time="2026-09-06T16:37:16.639Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9KZQPER5834EDRFG5BSH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:16.618" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1735gitlab # [ 388.762305] gitlab-workhorse[823]: time="2026-09-06T16:37:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131736gitlab # [ 388.766327] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 2331737gitlab # [ 388.770272] gitaly[893]: time="2026-09-06T16:37:16.640Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9KZQPER5834EDRFG5BSH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:16.618" grpc.time_ms=21.606 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1738gitlab # [ 388.957365] gitaly[893]: time="2026-09-06T16:37:16.901Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9M95JEVSD7485V4CB2JJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:16.865" pid=893 remote_ip="2001:db8:1::1" request_sha=b3d2c1931b289411f1db82d5f15e8ee183c22d9e response_bytes=223 span.kind=server system=grpc1739gitlab # [ 388.985804] gitaly[893]: time="2026-09-06T16:37:16.901Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=31 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1133 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=2 command.system_time_ms=23 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9M95JEVSD7485V4CB2JJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:16.865" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1740gitlab # [ 389.016540] gitlab-workhorse[823]: time="2026-09-06T16:37:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231741gitlab # [ 389.020126] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1931742gitlab # [ 389.024283] gitaly[893]: time="2026-09-06T16:37:16.902Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=31 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1133 command.oublock=0 command.real_time_ms=32 command.spawn_token_fork_ms=2 command.system_time_ms=23 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9M95JEVSD7485V4CB2JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:16.865" grpc.time_ms=36.893 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1743gitlab # [ 389.165522] gitaly[893]: time="2026-09-06T16:37:17.109Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9MHFDP5PZZYPSTRE324E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.090" grpc.time_ms=19.038 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1744gitlab # [ 389.193929] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1341745gitlab # [ 389.311708] gitaly[893]: time="2026-09-06T16:37:17.256Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9MNS6G7BXJA9QQ6934W3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.236" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1746gitlab # [ 389.332175] gitaly[893]: time="2026-09-06T16:37:17.256Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9MNS6G7BXJA9QQ6934W3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.236" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1747gitlab # [ 389.355503] gitlab-workhorse[823]: time="2026-09-06T16:37:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671748gitlab # [ 389.358512] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1441749gitlab # [ 389.364381] gitaly[893]: time="2026-09-06T16:37:17.256Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9MNS6G7BXJA9QQ6934W3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.236" grpc.time_ms=19.932 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1750gitlab # [ 389.491428] gitaly[893]: time="2026-09-06T16:37:17.435Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9MWYCXTVN0ATB999C9BT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.415" grpc.time_ms=19.606 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1751gitlab # [ 389.508781] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:17 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 931752gitlab # [ 389.662582] gitaly[893]: time="2026-09-06T16:37:17.606Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N00243WP2GXFEQDVH1Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.571" pid=893 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1753gitlab # [ 389.696637] gitaly[893]: time="2026-09-06T16:37:17.606Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=22 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N00243WP2GXFEQDVH1Q grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.571" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1754gitlab # [ 389.731261] gitlab-workhorse[823]: time="2026-09-06T16:37:17Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091755gitlab # [ 389.735013] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:17 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 1671756gitlab # [ 389.738780] gitaly[893]: time="2026-09-06T16:37:17.607Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=28 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=22 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N00243WP2GXFEQDVH1Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:17.571" grpc.time_ms=35.587 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1757gitlab # [ 390.122621] gitaly[893]: time="2026-09-06T16:37:18.065Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:37:48.063" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:37:18.064" grpc.time_ms=1.358 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1758gitlab # [ 390.244941] gitaly[893]: time="2026-09-06T16:37:18.189Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:37:48.169" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:18.169" grpc.time_ms=19.154 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1759gitlab # [ 390.302281] gitaly[893]: time="2026-09-06T16:37:18.246Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:37:28.226" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:18.227" grpc.time_ms=19 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1760gitlab # [ 390.344772] gitaly[893]: time="2026-09-06T16:37:18.289Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:37:28.019" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:18.270" grpc.time_ms=18.567 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1761gitlab # [ 390.511046] gitaly[893]: time="2026-09-06T16:37:18.455Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:37:28.203" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:18.454" grpc.time_ms=0.584 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1762gitlab # [ 390.582279] gitaly[893]: time="2026-09-06T16:37:18.526Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:37:28.257" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:18.508" grpc.time_ms=17.835 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1763gitlab # [ 390.695668] gitaly[893]: time="2026-09-06T16:37:18.639Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:37:18.639" grpc.time_ms=0.742 pid=893 span.kind=server system=grpc1764gitlab # [ 390.743232] gitaly[893]: time="2026-09-06T16:37:18.687Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:37:18.686" grpc.time_ms=0.957 pid=893 span.kind=server system=grpc1765gitlab # [ 390.773801] gitaly[893]: time="2026-09-06T16:37:18.703Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 diskcache=663b2dfe-677f-4d3d-80b1-fc475f9a4fc0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:37:28.008" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:18.568" pid=893 ref=refs/keep-around/8e238942deb4adcaba3415f1c90e6142f3fd1b72 remote_ip="2001:db8:1::1" revision=8e238942deb4adcaba3415f1c90e6142f3fd1b72 span.kind=server system=grpc username=root1766gitlab # [ 390.800811] gitaly[893]: time="2026-09-06T16:37:18.704Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=125 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5268 command.oublock=8 command.real_time_ms=126 command.spawn_token_fork_ms=2 command.system_time_ms=77 command.user_time_ms=48 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:37:28.008" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:37:18.568" grpc.time_ms=134.968 pid=893 ref=refs/keep-around/8e238942deb4adcaba3415f1c90e6142f3fd1b72 remote_ip="2001:db8:1::1" revision=8e238942deb4adcaba3415f1c90e6142f3fd1b72 span.kind=server system=grpc username=root1767gitlab # [ 390.824961] gitaly[893]: time="2026-09-06T16:37:18.725Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:37:48.723" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:37:18.724" grpc.time_ms=0.909 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768gitlab # [ 391.120344] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:19 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 13791769gitlab # [ 391.129397] nixbot[3813]: {"ts": "2026-09-06T16:37:19.065Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 \"HTTP/1.1 201 Created\""}1770gitlab # [ 391.141976] nixbot[3813]: {"ts": "2026-09-06T16:37:19.085Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-8e238942deb4-0"}1771gitlab # [ 391.364753] nixbot[3813]: {"ts": "2026-09-06T16:37:19.308Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-8e238942deb4-0", "argv0": "bwrap"}1772gitlab # [ 391.545265] systemd[1]: Started Nix Daemon.1773gitlab # [ 391.660335] nix-daemon[3985]: accepted connection from pid 3973, user nixbot1774gitlab # [ 396.526959] nix-daemon[3985]: accepted connection from pid 4032, user nixbot1775gitlab # [ 396.556634] nix-daemon[3985]: accepted connection from pid 3973, user nixbot1776gitlab # [ 401.488827] nix-daemon[3985]: accepted connection from pid 3973, user nixbot1777gitlab # [ 401.730049] gitaly[893]: time="2026-09-06T16:37:29.673Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA0T01BPCD36B9HYT4FF6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:37:59.662" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:37:29.663" grpc.time_ms=9.774 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1778gitlab # [ 401.963435] gitaly[893]: time="2026-09-06T16:37:29.907Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSA0T01BPCD36B9HYT4FF6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:37:39.208" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:29.889" grpc.time_ms=17.936 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1779gitlab # [ 402.083155] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 4881780gitlab # [ 402.092421] nixbot[3813]: {"ts": "2026-09-06T16:37:30.029Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 \"HTTP/1.1 201 Created\""}1781gitlab # [ 402.240801] gitaly[893]: time="2026-09-06T16:37:30.184Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA19PP9P4D0M0R0H24V71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:38:00.082" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:37:30.183" grpc.time_ms=0.963 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1782gitlab # [ 402.478913] gitaly[893]: time="2026-09-06T16:37:30.423Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M1VSA19PP9P4D0M0R0H24V71 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:37:40.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:30.404" grpc.time_ms=18.564 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1783gitlab # [ 402.625991] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 5311784gitlab # [ 402.629287] nixbot[3813]: {"ts": "2026-09-06T16:37:30.572Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 \"HTTP/1.1 201 Created\""}1785gitlab # [ 402.723945] gitaly[893]: time="2026-09-06T16:37:30.668Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA1V218E6MCPRNTJ33C0N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:38:00.565" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:37:30.666" grpc.time_ms=1.151 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1786gitlab # [ 402.830638] gitaly[893]: time="2026-09-06T16:37:30.773Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VSA1V218E6MCPRNTJ33C0N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:37:40.225" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:37:30.756" grpc.time_ms=17.403 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1787gitlab # [ 402.943439] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:30 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2921788gitlab # [ 402.951685] nixbot[3813]: {"ts": "2026-09-06T16:37:30.889Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 \"HTTP/1.1 201 Created\""}1789gitlab # [ 403.607970] gitaly[893]: time="2026-09-06T16:37:31.552Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=9 command.system_time_ms=5 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VSA2MF23VSVZ1549JKMNG6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:31.535" grpc.time_ms=15.764 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1790gitlab # [ 403.613215] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1471791gitlab # [ 403.769135] gitaly[893]: time="2026-09-06T16:37:31.713Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA2SDFCMDREDAA4NAV1B4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:31.688" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1792gitlab # [ 403.802368] gitaly[893]: time="2026-09-06T16:37:31.713Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=1 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA2SDFCMDREDAA4NAV1B4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:31.688" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1793gitlab # [ 403.833001] gitlab-workhorse[823]: time="2026-09-06T16:37:31Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671794gitlab # [ 403.836432] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:31 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1491795gitlab # [ 403.840544] gitaly[893]: time="2026-09-06T16:37:31.714Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=1 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA2SDFCMDREDAA4NAV1B4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:31.688" grpc.time_ms=25.892 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1796gitlab # [ 404.028844] gitaly[893]: time="2026-09-06T16:37:31.972Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VSA328JMZWKVD3HT373AB7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:31.952" grpc.time_ms=19.736 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1797gitlab # [ 404.051095] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:31 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1251798gitlab # [ 404.150408] gitaly[893]: time="2026-09-06T16:37:32.094Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA369FXM2MRXN6VC8NNHT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:32.073" pid=893 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1799gitlab # [ 404.177522] gitaly[893]: time="2026-09-06T16:37:32.094Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA369FXM2MRXN6VC8NNHT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:32.073" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1800gitlab # [ 404.203400] gitlab-workhorse[823]: time="2026-09-06T16:37:32Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671801gitlab # [ 404.206840] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:37:32 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1181802gitlab # [ 404.210256] gitaly[893]: time="2026-09-06T16:37:32.096Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSA369FXM2MRXN6VC8NNHT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:37:32.073" grpc.time_ms=21.776 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1803gitlab # [ 419.727604] sidekiq-cluster[3677]: 2026-09-06T16:37:47.670Z pid=3677 tid=y5x INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3677, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425443328}1804gitlab # [ 420.133744] sidekiq-cluster[3677]: W, [2026-09-06T16:37:48.077952 #3677] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1805gitlab # [ 420.201331] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:37:48.145Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1806gitlab # [ 423.420548] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:37:51.363Z","message":"GitLab reliable fetch activated!"}1807gitlab # [ 423.420973] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:37:51.363Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1808gitlab # [ 461.885376] postgres[4556]: [4556] ERROR: relation "table_without_models" does not exist1809gitlab # [ 461.886020] postgres[4556]: [4556] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1810gitlab # [ 466.987449] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:34.930Z","message":"Booted Rails 7.2.3.1 application in production environment"}1811gitlab # [ 466.989961] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:34.931Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1812gitlab # [ 466.996127] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:34.931Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1813gitlab # [ 466.998159] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:34.931Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1814gitlab # [ 467.000332] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:34.937Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1815gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 78.88 seconds)1816gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1817gitlab # [ 467.375870] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1291818gitlab # [ 467.550302] gitaly[893]: time="2026-09-06T16:38:35.492Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC11RBN8GG5VEQX0TV6G0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:35.473" grpc.time_ms=18.671 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1819gitlab # Enumerating objects: 5, done.1820gitlab # [ 467.590631] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:35 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 1721821gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1822gitlab # Delta compression using up to 4 threads1823gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1824gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 68.00 KiB/s, done.1825gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1826gitlab # [ 468.034932] gitaly[893]: time="2026-09-06T16:38:35.979Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-06T16:39:05.058" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-06T16:38:35.958" grpc.time_ms=20.221 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1827gitlab # [ 468.076095] gitaly[893]: time="2026-09-06T16:38:36.017Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX duration_ms=210ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/allowed"1828gitlab # [ 468.114568] gitaly[893]: time="2026-09-06T16:38:36.058Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX duration_ms=40ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/pre_receive"1829gitlab # [ 468.123740] gitaly[893]: time="2026-09-06T16:38:36.059Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5315 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:35.805" grpc.time_ms=253.454 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1830gitlab # [ 468.186831] gitaly[893]: time="2026-09-06T16:38:36.131Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:36.128" grpc.time_ms=1.189 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1831gitlab # [ 468.238144] gitaly[893]: time="2026-09-06T16:38:36.182Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:36.180" grpc.time_ms=1.188 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1832gitlab # [ 468.286767] gitaly[893]: time="2026-09-06T16:38:36.230Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:36.229" grpc.time_ms=1.018 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1833gitlab # [ 468.529788] gitaly[893]: time="2026-09-06T16:38:36.473Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX duration_ms=192ns method=POST pid=893 status=200 url="http://unix/api/v4/internal/post_receive"1834gitlab # [ 468.540908] gitaly[893]: time="2026-09-06T16:38:36.475Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4809 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:36.276" grpc.time_ms=198.912 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1835gitlab # [ 468.568151] gitaly[893]: time="2026-09-06T16:38:36.511Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX diskcache=9a4dd1ac-0126-41a2-b37c-1591fa51ff6d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:35.689" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1836gitlab # [ 468.586081] gitaly[893]: time="2026-09-06T16:38:36.512Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=356 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13352 command.oublock=32 command.real_time_ms=800 command.spawn_token_fork_ms=2 command.system_time_ms=223 command.user_time_ms=133 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:35.689" grpc.time_ms=822.387 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1837gitlab # To http://gitlab/root/test-flake.git1838gitlab # [ 468.606654] gitlab-workhorse[823]: time="2026-09-06T16:38:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521839gitlab # 8e23894..4061fd5 [ 468.609649] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:36 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 9481840gitlab # master -> master1841gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.48 seconds)1842gitlab: must succeed: git -C /tmp/test-flake rev-parse master1843gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.03 seconds)1844gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1845gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.15 seconds)1846gitlab # [ 468.803852] nixbot[3813]: INFO: ::ffff:127.0.0.1:60744 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1847gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1848gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.09 seconds)1849gitlab # [ 469.887714] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:37.830Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"6f11d6b9d7a80f97a130358d","created_at":"2026-09-06T16:37:02.739Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:02.811Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-6f11d6b9d7a80f97a130358d: start","job_status":"start","queue_duration_s":95.019191,"scheduling_latency_s":95.019251}1850gitlab # [ 469.889274] nixbot[3813]: INFO: ::ffff:127.0.0.1:60746 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1851gitlab # [ 470.193481] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.136Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"6f11d6b9d7a80f97a130358d","created_at":"2026-09-06T16:37:02.739Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:02.811Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-6f11d6b9d7a80f97a130358d: done: 0.306448 sec","job_status":"done","queue_duration_s":95.019191,"scheduling_latency_s":95.019251,"redis_calls":23,"redis_duration_s":0.010099,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.005162,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002862,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.17610912,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.306448,"completed_at":"2026-09-06T16:38:38.136Z","load_balancing_strategy":"primary","db_duration_s":0.099582,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1852gitlab # [ 470.288336] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.139Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"2a58883cc427744ca7794321","created_at":"2026-09-06T16:37:02.833Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:02.842Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-2a58883cc427744ca7794321: start","job_status":"start","queue_duration_s":95.29727,"scheduling_latency_s":95.297307}1853gitlab # [ 470.367895] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.311Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"2a58883cc427744ca7794321","created_at":"2026-09-06T16:37:02.833Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:02.842Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-2a58883cc427744ca7794321: done: 0.171619 sec","job_status":"done","queue_duration_s":95.29727,"scheduling_latency_s":95.297307,"redis_calls":14,"redis_duration_s":0.005018,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001138,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002354,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001526,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.126301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13032536,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.171619,"completed_at":"2026-09-06T16:38:38.311Z","load_balancing_strategy":"primary","db_duration_s":0.034477,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1854gitlab # [ 470.459975] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.313Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"4acb0c022c778b61f4efa151","created_at":"2026-09-06T16:37:03.636Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:03.649Z","job_size_bytes":5,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-4acb0c022c778b61f4efa151: start","job_status":"start","queue_duration_s":94.664314,"scheduling_latency_s":94.664343}1855gitlab # [ 470.594919] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.538Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"4acb0c022c778b61f4efa151","created_at":"2026-09-06T16:37:03.636Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:03.649Z","job_size_bytes":5,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-4acb0c022c778b61f4efa151: done: 0.22483 sec","job_status":"done","queue_duration_s":94.664314,"scheduling_latency_s":94.664343,"redis_calls":12,"redis_duration_s":0.003552,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000808,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002221,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000523,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.173096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003606,"duration_s":0.22483,"completed_at":"2026-09-06T16:38:38.538Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.04498,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1856gitlab # [ 470.694760] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.540Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ada9cf52e25d8933531f61e5","created_at":"2026-09-06T16:37:03.699Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:03.709Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ada9cf52e25d8933531f61e5: start","job_status":"start","queue_duration_s":94.831737,"scheduling_latency_s":94.831761}1857gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1858gitlab # [ 470.890595] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.833Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"ada9cf52e25d8933531f61e5","created_at":"2026-09-06T16:37:03.699Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:03.709Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-ada9cf52e25d8933531f61e5: done: 0.292181 sec","job_status":"done","queue_duration_s":94.831737,"scheduling_latency_s":94.831761,"redis_calls":11,"redis_duration_s":0.003676,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001028,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001989,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000659,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.247,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.247,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.177986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060284,"duration_s":0.292181,"completed_at":"2026-09-06T16:38:38.832Z","load_balancing_strategy":"primary","db_duration_s":0.117829,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1859gitlab # [ 471.000082] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.835Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"230b70095b213392bc8e25ed","created_at":"2026-09-06T16:37:03.715Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:03.724Z","job_size_bytes":89,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-230b70095b213392bc8e25ed: start","job_status":"start","queue_duration_s":95.111573,"scheduling_latency_s":95.111597}1860gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.13 seconds)1861gitlab # [ 471.023486] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.888Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"230b70095b213392bc8e25ed","created_at":"2026-09-06T16:37:03.715Z","trace_propagation_headers":{"sentry-trace":"83f599af8232463689cef835ff8083eb-70ef62301f1e4b92","baggage":"sentry-trace_id=83f599af8232463689cef835ff8083eb,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS92QMD40TK476DKRDFEBA","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:03.724Z","job_size_bytes":89,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-230b70095b213392bc8e25ed: done: 0.052674 sec","job_status":"done","queue_duration_s":95.111573,"scheduling_latency_s":95.111597,"redis_calls":11,"redis_duration_s":0.0031999999999999997,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000788,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00053102,"duration_s":0.052674,"completed_at":"2026-09-06T16:38:38.888Z","load_balancing_strategy":"primary","db_duration_s":0.010441,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1862gitlab # [ 471.079708] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.890Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8e238942deb4adcaba3415f1c90e6142f3fd1b72 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3f28f4e40f680fe88305f333","created_at":"2026-09-06T16:37:11.457Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-78b2d280e0974f84","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:37:11.493Z","job_size_bytes":185,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3f28f4e40f680fe88305f333: start","job_status":"start","queue_duration_s":87.396852,"scheduling_latency_s":87.396875}1863gitlab # [ 471.124114] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:38.914Z","message":"changes: \"0000000000000000000000000000000000000000 8e238942deb4adcaba3415f1c90e6142f3fd1b72 refs/heads/master\\n\""}1864gitlab # [ 471.129565] gitaly[893]: time="2026-09-06T16:38:38.962Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:48.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:38.952" grpc.time_ms=10.088 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab # [ 471.150994] nixbot[3813]: INFO: ::ffff:127.0.0.1:42736 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1866gitlab # [ 471.154232] gitaly[893]: time="2026-09-06T16:38:38.987Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:38:48.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:38.969" grpc.time_ms=17.614 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 471.173122] gitaly[893]: time="2026-09-06T16:38:39.064Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:38:49.008" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:39.049" grpc.time_ms=14.344 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1868gitlab # [ 471.191897] gitaly[893]: time="2026-09-06T16:38:39.123Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3936 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:39.122" grpc.time_ms=1 pid=893 span.kind=server system=grpc1869gitlab # [ 471.222099] gitaly[893]: time="2026-09-06T16:38:39.166Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3938 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:39.165" grpc.time_ms=0.798 pid=893 span.kind=server system=grpc1870gitlab # [ 471.238484] gitaly[893]: time="2026-09-06T16:38:39.180Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 diskcache=8afc3357-1348-4dae-9352-3f0355c44d06 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:38:49.037" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:39.078" pid=893 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1871gitlab # [ 471.254199] gitaly[893]: time="2026-09-06T16:38:39.181Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=108 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5047 command.oublock=8 command.real_time_ms=95 command.spawn_token_fork_ms=1 command.system_time_ms=52 command.user_time_ms=56 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:38:49.037" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:39.078" grpc.time_ms=102.425 pid=893 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1872gitlab # [ 471.270900] gitaly[893]: time="2026-09-06T16:38:39.200Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-06T16:38:49.159" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:39.200" grpc.time_ms=0.331 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1873gitlab # [ 471.286125] gitaly[893]: time="2026-09-06T16:38:39.230Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=20 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-06T16:38:49.165" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:39.206" grpc.time_ms=23.817 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1874gitlab # [ 471.425684] gitaly[893]: time="2026-09-06T16:38:39.369Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-06T16:39:09.056" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:39.356" grpc.time_ms=12.635 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1875gitlab # [ 471.489185] gitaly[893]: time="2026-09-06T16:38:39.433Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:38:49.001" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:39.432" grpc.time_ms=0.665 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1876gitlab # [ 471.517921] gitaly[893]: time="2026-09-06T16:38:39.449Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:38:49.007" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:38:39.437" grpc.time_ms=11.422 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1877gitlab # [ 471.553485] gitaly[893]: time="2026-09-06T16:38:39.497Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:49.048" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:39.479" grpc.time_ms=18.029 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1878gitlab # [ 471.583024] gitaly[893]: time="2026-09-06T16:38:39.527Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:49.078" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:39.509" grpc.time_ms=17.856 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1879gitlab # [ 471.637785] gitaly[893]: time="2026-09-06T16:38:39.581Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=2 command.system_time_ms=18 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:39:09.251" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:39.551" grpc.time_ms=29.832 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1881gitlab # [ 472.086722] nixbot[3813]: INFO: ::ffff:127.0.0.1:42742 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1882gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.07 seconds)1883gitlab # [ 472.262094] gitaly[893]: time="2026-09-06T16:38:40.205Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:38:50.161" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:40.202" grpc.time_ms=2.571 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1884gitlab # [ 472.304677] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:40.247Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 8e238942deb4adcaba3415f1c90e6142f3fd1b72 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"3f28f4e40f680fe88305f333","created_at":"2026-09-06T16:37:11.457Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-78b2d280e0974f84","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:37:11.493Z","job_size_bytes":185,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-3f28f4e40f680fe88305f333: done: 1.356831 sec","job_status":"done","queue_duration_s":87.396852,"scheduling_latency_s":87.396875,"gitaly_calls":13,"gitaly_duration_s":0.316632,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.066458,"redis_read_bytes":2103,"redis_write_bytes":21746,"redis_cache_calls":15,"redis_cache_duration_s":0.00628,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.010143,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002224,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7834,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.009922,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.022893,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.014996,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.093,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.329,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.372,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.093,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.879071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00092314,"exclusive_lock_hold_duration_s":0.03417996,"duration_s":1.356831,"completed_at":"2026-09-06T16:38:40.247Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.162078,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1885gitlab # [ 472.398773] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:40.251Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a95786b843ba205b3d77bd5f","created_at":"2026-09-06T16:37:18.737Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-344bcba297114d91","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:18.753Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a95786b843ba205b3d77bd5f: start","job_status":"start","queue_duration_s":81.498072,"scheduling_latency_s":81.498129}1886gitlab # [ 472.507460] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:40.450Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"a95786b843ba205b3d77bd5f","created_at":"2026-09-06T16:37:18.737Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-344bcba297114d91","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:18.753Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-a95786b843ba205b3d77bd5f: done: 0.199487 sec","job_status":"done","queue_duration_s":81.498072,"scheduling_latency_s":81.498129,"redis_calls":16,"redis_duration_s":0.007809,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.00397,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000963,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.144919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0009226,"duration_s":0.199487,"completed_at":"2026-09-06T16:38:40.450Z","load_balancing_strategy":"primary","db_duration_s":0.040581,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1887gitlab # [ 472.599904] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:40.454Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2c61da0837ca7e7360ed1e95","created_at":"2026-09-06T16:37:18.755Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-344bcba297114d91","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:18.768Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2c61da0837ca7e7360ed1e95: start","job_status":"start","queue_duration_s":81.685593,"scheduling_latency_s":81.685636}1888gitlab # [ 472.720070] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:40.663Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"2c61da0837ca7e7360ed1e95","created_at":"2026-09-06T16:37:18.755Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-344bcba297114d91","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:18.768Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-2c61da0837ca7e7360ed1e95: done: 0.208909 sec","job_status":"done","queue_duration_s":81.685593,"scheduling_latency_s":81.685636,"redis_calls":21,"redis_duration_s":0.009204,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.004137,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002406,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002661,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.036,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.137503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00081444,"duration_s":0.208909,"completed_at":"2026-09-06T16:38:40.663Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.058192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1889gitlab # [ 472.816228] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:40.666Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"20b73abd378a3c5b19a8b6aa","created_at":"2026-09-06T16:37:19.017Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-344bcba297114d91","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:19.030Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-20b73abd378a3c5b19a8b6aa: start","job_status":"start","queue_duration_s":81.6359,"scheduling_latency_s":81.635942}1890gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1891gitlab # [ 473.150778] nixbot[3813]: INFO: ::ffff:127.0.0.1:42754 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1892gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.06 seconds)1893gitlab # [ 473.198976] gitaly[893]: time="2026-09-06T16:38:41.143Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:41.142" grpc.time_ms=0.895 pid=893 span.kind=server system=grpc1894gitlab # [ 473.255542] gitaly[893]: time="2026-09-06T16:38:41.199Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:41.198" grpc.time_ms=0.995 pid=893 span.kind=server system=grpc1895gitlab # [ 473.317103] gitaly[893]: time="2026-09-06T16:38:41.260Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:41.258" grpc.time_ms=0.995 pid=893 span.kind=server system=grpc1896gitlab # [ 473.345275] gitaly[893]: time="2026-09-06T16:38:41.277Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 diskcache=38b44d75-7e5b-4166-9b57-8db5f210dada grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:39:11.080" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:41.081" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1897gitlab # [ 473.366367] gitaly[893]: time="2026-09-06T16:38:41.278Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=216 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7507 command.oublock=0 command.real_time_ms=189 command.spawn_token_fork_ms=1 command.system_time_ms=115 command.user_time_ms=101 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:39:11.080" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:41.081" grpc.time_ms=196.378 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1898gitlab # [ 473.569744] gitaly[893]: time="2026-09-06T16:38:41.513Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:39:11.211" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:41.511" grpc.time_ms=2.003 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899gitlab # [ 473.629810] gitaly[893]: time="2026-09-06T16:38:41.574Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9N7K00CZWZ08Z8R32H84 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:11.271" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:41.572" grpc.time_ms=1.667 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1900gitlab # [ 473.669813] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:41.599Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"20b73abd378a3c5b19a8b6aa","created_at":"2026-09-06T16:37:19.017Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-344bcba297114d91","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:37:19.030Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-20b73abd378a3c5b19a8b6aa: done: 0.933329 sec","job_status":"done","queue_duration_s":81.6359,"scheduling_latency_s":81.635942,"gitaly_calls":3,"gitaly_duration_s":0.212664,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.042259,"redis_read_bytes":455,"redis_write_bytes":27997,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001916,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00319,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.014922,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005926,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14491,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.008483,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.001331,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000985,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00359,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.106,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.038,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.044,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.052,"db_ci_duration_s":0.055,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.585112,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.8311978800000001,"duration_s":0.933329,"completed_at":"2026-09-06T16:38:41.599Z","load_balancing_strategy":"primary","db_duration_s":0.125314,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1901gitlab # [ 473.747870] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:41.602Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8e238942deb4adcaba3415f1c90e6142f3fd1b72 4061fd525383d96028769d7d55bafd8ab2bd0ace refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"0f706d59b9d17bc02f7a07cb","created_at":"2026-09-06T16:38:36.390Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-b3e57eba0b954d16","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:36.401Z","job_size_bytes":185,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-0f706d59b9d17bc02f7a07cb: start","job_status":"start","queue_duration_s":5.200478,"scheduling_latency_s":5.200509}1902gitlab # [ 473.765677] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:41.619Z","message":"changes: \"8e238942deb4adcaba3415f1c90e6142f3fd1b72 4061fd525383d96028769d7d55bafd8ab2bd0ace refs/heads/master\\n\""}1903gitlab # [ 473.768265] gitaly[893]: time="2026-09-06T16:38:41.662Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:51.212" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:41.643" grpc.time_ms=18.947 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 473.781931] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:41.693Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"6aa6c5da793999093cba030f","created_at":"2026-09-06T16:38:41.691Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-03211541cfc643d8","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"f5a461bc1854befe30b6d11d","job_size_bytes":46,"pid":3677,"job_status":"deduplicated","message":"ProjectCacheWorker JID-6aa6c5da793999093cba030f: deduplicated: until executing","deduplication.type":"until executing"}1905gitlab # [ 473.797437] gitaly[893]: time="2026-09-06T16:38:41.687Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-06T16:38:51.238" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:41.669" grpc.time_ms=18.068 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1906gitlab # [ 473.811370] gitaly[893]: time="2026-09-06T16:38:41.731Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-06T16:38:51.282" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:41.713" grpc.time_ms=18.284 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1907gitlab # [ 473.825472] gitaly[893]: time="2026-09-06T16:38:41.759Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-06T16:39:11.437" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:41.738" grpc.time_ms=21.254 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 473.844312] gitaly[893]: time="2026-09-06T16:38:41.788Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-06T16:38:51.005" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-06T16:38:41.776" grpc.time_ms=11.897 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1909gitlab # [ 473.882662] gitaly[893]: time="2026-09-06T16:38:41.818Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:51.031" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:41.802" grpc.time_ms=15.939 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1910gitlab # [ 474.048934] gitaly[893]: time="2026-09-06T16:38:41.993Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-06T16:38:51.219" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:41.990" grpc.time_ms=2.723 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1912gitlab # [ 474.099868] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.042Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","8e238942deb4adcaba3415f1c90e6142f3fd1b72 4061fd525383d96028769d7d55bafd8ab2bd0ace refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"0f706d59b9d17bc02f7a07cb","created_at":"2026-09-06T16:38:36.390Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-b3e57eba0b954d16","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:36.401Z","job_size_bytes":185,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-0f706d59b9d17bc02f7a07cb: done: 0.44043 sec","job_status":"done","queue_duration_s":5.200478,"scheduling_latency_s":5.200509,"gitaly_calls":7,"gitaly_duration_s":0.134602,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.031192,"redis_read_bytes":501,"redis_write_bytes":11944,"redis_cache_calls":9,"redis_cache_duration_s":0.003195,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001559,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5721,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.004548,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.010064,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.011826,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.037,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.037,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.259797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00095102,"exclusive_lock_hold_duration_s":0.03188224,"duration_s":0.44043,"completed_at":"2026-09-06T16:38:42.042Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044749,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1913gitlab # [ 474.197285] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.046Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f5a461bc1854befe30b6d11d","created_at":"2026-09-06T16:38:38.990Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.012Z","job_size_bytes":46,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f5a461bc1854befe30b6d11d: start","job_status":"start","queue_duration_s":3.033538,"scheduling_latency_s":3.033585}1914gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.07 seconds)1915gitlab # [ 474.218927] gitaly[893]: time="2026-09-06T16:38:42.145Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-06T22:38:42.137" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:42.138" grpc.time_ms=6.253 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1916gitlab # [ 474.231533] gitaly[893]: time="2026-09-06T16:38:42.173Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:39:12.158" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:42.159" grpc.time_ms=13.824 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1917gitlab # [ 474.247071] nixbot[3813]: INFO: ::ffff:127.0.0.1:42760 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1918gitlab # [ 474.270143] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.213Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"f5a461bc1854befe30b6d11d","created_at":"2026-09-06T16:38:38.990Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.012Z","job_size_bytes":46,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f5a461bc1854befe30b6d11d: done: 0.167232 sec","job_status":"done","queue_duration_s":3.033538,"scheduling_latency_s":3.033585,"gitaly_calls":2,"gitaly_duration_s":0.02798,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.010623,"redis_read_bytes":69,"redis_write_bytes":4887,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002191,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000818,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002309,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.004061,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044206,"duration_s":0.167232,"completed_at":"2026-09-06T16:38:42.213Z","load_balancing_strategy":"primary","db_duration_s":0.023467,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1919gitlab # [ 474.330310] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.216Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"eaaa5b41a0a858c077310294","created_at":"2026-09-06T16:38:39.036Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.043Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-eaaa5b41a0a858c077310294: start","job_status":"start","queue_duration_s":3.17263,"scheduling_latency_s":3.172658}1920gitlab # [ 474.349906] gitaly[893]: time="2026-09-06T16:38:42.283Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 diskcache=ff76b376-48f0-4b67-a327-f8374f9eeab1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:38:42.250" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:42.251" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1921gitlab # [ 474.360762] gitaly[893]: time="2026-09-06T16:38:42.283Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=34 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:38:42.250" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:42.251" grpc.time_ms=32.057 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1922gitlab # [ 474.430497] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.373Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"eaaa5b41a0a858c077310294","created_at":"2026-09-06T16:38:39.036Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.043Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-eaaa5b41a0a858c077310294: done: 0.15782 sec","job_status":"done","queue_duration_s":3.17263,"scheduling_latency_s":3.172658,"gitaly_calls":1,"gitaly_duration_s":0.036178,"redis_calls":15,"redis_duration_s":0.005743,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000605,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000869,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001816,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.046,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11470374000000001,"duration_s":0.15782,"completed_at":"2026-09-06T16:38:42.373Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1923gitlab # [ 474.513117] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.376Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a325d43ffbd57488b035a23a","created_at":"2026-09-06T16:38:39.193Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.196Z","job_size_bytes":23,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a325d43ffbd57488b035a23a: start","job_status":"start","queue_duration_s":3.180569,"scheduling_latency_s":3.180606}1924gitlab # [ 474.543207] gitaly[893]: time="2026-09-06T16:38:42.444Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-06T16:39:12.125" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:42.426" grpc.time_ms=18.016 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1925gitlab # [ 474.562356] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.467Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"a325d43ffbd57488b035a23a","created_at":"2026-09-06T16:38:39.193Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.196Z","job_size_bytes":23,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-a325d43ffbd57488b035a23a: done: 0.090806 sec","job_status":"done","queue_duration_s":3.180569,"scheduling_latency_s":3.180606,"gitaly_calls":1,"gitaly_duration_s":0.021268,"redis_calls":16,"redis_duration_s":0.006164,"redis_read_bytes":52,"redis_write_bytes":2047,"redis_queues_calls":1,"redis_queues_duration_s":0.000661,"redis_queues_read_bytes":1,"redis_queues_write_bytes":978,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002756,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090806,"completed_at":"2026-09-06T16:38:42.467Z","load_balancing_strategy":"primary","db_duration_s":0.005148,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1926gitlab # [ 474.611752] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:42.470Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a720ac6d51814249c9b46f84","created_at":"2026-09-06T16:38:39.983Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9085ec1af075a1b621878af5889029ea0eda90be217e577c337df69a045652b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.993Z","job_size_bytes":467,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a720ac6d51814249c9b46f84: start","job_status":"start","queue_duration_s":2.476874,"scheduling_latency_s":2.476917}1927gitlab # [ 474.629263] gitaly[893]: time="2026-09-06T16:38:42.529Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:12.227" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:42.528" grpc.time_ms=1.118 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1928gitlab # [ 474.641711] gitaly[893]: time="2026-09-06T16:38:42.551Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:38:52.103" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:42.533" grpc.time_ms=17.246 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1929gitlab # [ 474.825776] gitaly[893]: time="2026-09-06T16:38:42.770Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:39:12.468" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:42.768" grpc.time_ms=1.465 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1930gitlab # [ 474.864345] gitaly[893]: time="2026-09-06T16:38:42.779Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=31 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:39:12.474" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:42.775" grpc.time_ms=4.321 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1931gitlab # [ 474.965650] gitaly[893]: time="2026-09-06T16:38:42.909Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:52.121" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:42.892" grpc.time_ms=17.311 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1932gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1933gitlab # [ 475.294657] nixbot[3813]: INFO: ::ffff:127.0.0.1:42764 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1934gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.07 seconds)1935gitlab # [ 475.909491] gitaly[893]: time="2026-09-06T16:38:43.852Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=16 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:39:13.523" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:43.824" grpc.time_ms=28.18 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1936gitlab # [ 475.951184] gitaly[893]: time="2026-09-06T16:38:43.888Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VS9E6B5EHZBCHH9HP0K518 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:39:13.558" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:43.859" grpc.time_ms=28.35 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1937gitlab # [ 476.017342] sidekiq-cluster[3677]: {"severity":"WARN","time":"2026-09-06T16:38:43.960Z","class":"CreatePipelineWorker","correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"8e238942deb4adcaba3415f1c90e6142f3fd1b72","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"8e238942deb4adcaba3415f1c90e6142f3fd1b72","ref":"refs/heads/master"},"retry":0}1938gitlab # [ 476.039399] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:43.978Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a720ac6d51814249c9b46f84","created_at":"2026-09-06T16:38:39.983Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9085ec1af075a1b621878af5889029ea0eda90be217e577c337df69a045652b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:39.993Z","job_size_bytes":467,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a720ac6d51814249c9b46f84: done: 1.50797 sec","job_status":"done","queue_duration_s":2.476874,"scheduling_latency_s":2.476917,"gitaly_calls":7,"gitaly_duration_s":0.124763,"redis_calls":51,"redis_duration_s":0.021786,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.00165,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.012606,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0025,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003144,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.053,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.285562,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069536,"duration_s":1.50797,"completed_at":"2026-09-06T16:38:43.978Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.095281,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1939gitlab # [ 476.142866] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:43.983Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"1612dbeb-e087-4f5f-ab9f-492158d97630\"}"],"class":"WebHookWorker","jid":"46fb3e19b9873e6d1bc76f54","created_at":"2026-09-06T16:38:40.154Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b19b2c952b5d99a523bbcc86033c680b867782848a899f85304b6d471f335e76","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:40.162Z","job_size_bytes":1751,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-46fb3e19b9873e6d1bc76f54: start","job_status":"start","queue_duration_s":3.820654,"scheduling_latency_s":3.820709}1940gitlab # [ 476.285312] nixbot[3813]: INFO: ::ffff:127.0.0.1:42776 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1941gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1942gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.06 seconds)1943gitlab # [ 476.339802] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.282Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"1612dbeb-e087-4f5f-ab9f-492158d97630\"}"],"class":"WebHookWorker","jid":"46fb3e19b9873e6d1bc76f54","created_at":"2026-09-06T16:38:40.154Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-e9b19434690b47f2","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:b19b2c952b5d99a523bbcc86033c680b867782848a899f85304b6d471f335e76","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:40.162Z","job_size_bytes":1751,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-46fb3e19b9873e6d1bc76f54: done: 0.298987 sec","job_status":"done","queue_duration_s":3.820654,"scheduling_latency_s":3.820709,"redis_calls":19,"redis_duration_s":0.009264,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002028,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000953,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.004205,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002078,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09740977999996403,"cpu_s":0.171217,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00163762,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.298987,"completed_at":"2026-09-06T16:38:44.282Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007019,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1944gitlab # [ 476.435090] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.287Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c07507db83f19536febc582f","created_at":"2026-09-06T16:38:41.039Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.046Z","job_size_bytes":23,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c07507db83f19536febc582f: start","job_status":"start","queue_duration_s":3.240292,"scheduling_latency_s":3.240344}1945gitlab # [ 476.454705] nixbot[3813]: INFO: ::ffff:127.0.0.1:42790 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1946gitlab # [ 476.538361] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.481Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"c07507db83f19536febc582f","created_at":"2026-09-06T16:38:41.039Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.046Z","job_size_bytes":23,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-c07507db83f19536febc582f: done: 0.194133 sec","job_status":"done","queue_duration_s":3.240292,"scheduling_latency_s":3.240344,"redis_calls":10,"redis_duration_s":0.00636,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000752,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004547,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001061,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.138633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00113688,"duration_s":0.194133,"completed_at":"2026-09-06T16:38:44.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038024,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1947gitlab # [ 476.628989] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.486Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d0c8b5e6e20b6641955d7774","created_at":"2026-09-06T16:38:41.290Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.304Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d0c8b5e6e20b6641955d7774: start","job_status":"start","queue_duration_s":3.181837,"scheduling_latency_s":3.181902}1948gitlab # [ 476.648903] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:44 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 2221949gitlab # [ 476.651172] gitaly[893]: time="2026-09-06T16:38:44.523Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=4 command.system_time_ms=8 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC9TE88252MJYC0P3W4NX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:44.506" grpc.time_ms=16.811 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1950gitlab # [ 476.665768] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.585Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d0c8b5e6e20b6641955d7774","created_at":"2026-09-06T16:38:41.290Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.304Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d0c8b5e6e20b6641955d7774: done: 0.10021 sec","job_status":"done","queue_duration_s":3.181837,"scheduling_latency_s":3.181902,"redis_calls":10,"redis_duration_s":0.004931,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.071783,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00097606,"duration_s":0.10021,"completed_at":"2026-09-06T16:38:44.585Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1951gitlab # [ 476.713447] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.589Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e09ad7708a7e3ba29bbef526","created_at":"2026-09-06T16:38:41.333Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.349Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e09ad7708a7e3ba29bbef526: start","job_status":"start","queue_duration_s":3.240002,"scheduling_latency_s":3.24005}1952gitlab # [ 476.731516] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.674Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e09ad7708a7e3ba29bbef526","created_at":"2026-09-06T16:38:41.333Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.349Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e09ad7708a7e3ba29bbef526: done: 0.085339 sec","job_status":"done","queue_duration_s":3.240002,"scheduling_latency_s":3.24005,"redis_calls":10,"redis_duration_s":0.003745,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000773,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00077,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.058605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000575,"duration_s":0.085339,"completed_at":"2026-09-06T16:38:44.674Z","load_balancing_strategy":"primary","db_duration_s":0.013921,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1953gitlab # [ 476.778523] gitlab-workhorse[823]: time="2026-09-06T16:38:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671954gitlab # [ 476.780549] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1221955gitlab # [ 476.782576] gitaly[893]: time="2026-09-06T16:38:44.649Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCA1FJDVD6RFNXAZCFJZ3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:44.629" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1956gitlab # [ 476.793537] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.677Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b486828af7a97fadcd52ea6d","created_at":"2026-09-06T16:38:41.381Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.393Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b486828af7a97fadcd52ea6d: start","job_status":"start","queue_duration_s":3.28398,"scheduling_latency_s":3.284015}1957gitlab # [ 476.810221] gitaly[893]: time="2026-09-06T16:38:44.649Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCA1FJDVD6RFNXAZCFJZ3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:44.629" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1958gitlab # [ 476.824804] gitaly[893]: time="2026-09-06T16:38:44.650Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCA1FJDVD6RFNXAZCFJZ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:44.629" grpc.time_ms=20.09 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1959gitlab # [ 476.840629] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.754Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b486828af7a97fadcd52ea6d","created_at":"2026-09-06T16:38:41.381Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.393Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b486828af7a97fadcd52ea6d: done: 0.076795 sec","job_status":"done","queue_duration_s":3.28398,"scheduling_latency_s":3.284015,"redis_calls":13,"redis_duration_s":0.005056,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000881,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000939,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00071686,"duration_s":0.076795,"completed_at":"2026-09-06T16:38:44.754Z","load_balancing_strategy":"primary","db_duration_s":0.013165,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1960gitlab # [ 476.889175] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.758Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bf99ac6dd7bf9d8545883322","created_at":"2026-09-06T16:38:41.394Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.404Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bf99ac6dd7bf9d8545883322: start","job_status":"start","queue_duration_s":3.353321,"scheduling_latency_s":3.353359}1961gitlab # [ 476.907044] gitlab-workhorse[823]: time="2026-09-06T16:38:44Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3361962gitlab # [ 476.909149] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:44 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 1161963gitlab # [ 476.911613] gitaly[893]: time="2026-09-06T16:38:44.769Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCA5DFR36JCP0W6BA7BSV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:44.738" pid=893 remote_ip="2001:db8:1::1" request_sha=d5d2eef1eeb153136b30116c5c5d9e2998cda50a response_bytes=336 span.kind=server system=grpc1964gitlab # [ 476.923763] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.827Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"bf99ac6dd7bf9d8545883322","created_at":"2026-09-06T16:38:41.394Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.404Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-bf99ac6dd7bf9d8545883322: done: 0.069207 sec","job_status":"done","queue_duration_s":3.353321,"scheduling_latency_s":3.353359,"redis_calls":10,"redis_duration_s":0.004451,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001026,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000732,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054046,"duration_s":0.069207,"completed_at":"2026-09-06T16:38:44.827Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 476.971746] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.830Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c4162bb22968984460036264","created_at":"2026-09-06T16:38:41.406Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.417Z","job_size_bytes":19,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c4162bb22968984460036264: start","job_status":"start","queue_duration_s":3.412497,"scheduling_latency_s":3.412562}1966gitlab # [ 476.989214] nixbot[3813]: {"ts": "2026-09-06T16:38:44.840Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1967gitlab # [ 476.991832] gitaly[893]: time="2026-09-06T16:38:44.769Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=1 command.system_time_ms=19 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCA5DFR36JCP0W6BA7BSV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:44.738" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1968gitlab # [ 477.006715] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.886Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"c4162bb22968984460036264","created_at":"2026-09-06T16:38:41.406Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.417Z","job_size_bytes":19,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-c4162bb22968984460036264: done: 0.056305 sec","job_status":"done","queue_duration_s":3.412497,"scheduling_latency_s":3.412562,"redis_calls":11,"redis_duration_s":0.004226,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000568,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002549,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049344,"duration_s":0.056305,"completed_at":"2026-09-06T16:38:44.886Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1969gitlab # [ 477.050275] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.889Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2d1606a481989d4601ba839c","created_at":"2026-09-06T16:38:41.423Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.432Z","job_size_bytes":66,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2d1606a481989d4601ba839c: start","job_status":"start","queue_duration_s":3.456847,"scheduling_latency_s":3.456884}1970gitlab # [ 477.068307] gitaly[893]: time="2026-09-06T16:38:44.769Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=1 command.system_time_ms=19 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCA5DFR36JCP0W6BA7BSV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:44.738" grpc.time_ms=31.32 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc1971gitlab # [ 477.083770] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.954Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"2d1606a481989d4601ba839c","created_at":"2026-09-06T16:38:41.423Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.432Z","job_size_bytes":66,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-2d1606a481989d4601ba839c: done: 0.064688 sec","job_status":"done","queue_duration_s":3.456847,"scheduling_latency_s":3.456884,"redis_calls":10,"redis_duration_s":0.004912000000000001,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00079,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00223,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001892,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069602,"duration_s":0.064688,"completed_at":"2026-09-06T16:38:44.954Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1972gitlab # [ 477.130872] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:44.956Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8913dd35b497e7f969cfcb69","created_at":"2026-09-06T16:38:41.434Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.443Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8913dd35b497e7f969cfcb69: start","job_status":"start","queue_duration_s":3.512894,"scheduling_latency_s":3.512922}1973gitlab # [ 477.146987] gitaly[893]: time="2026-09-06T16:38:44.983Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:14.080" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:44.981" grpc.time_ms=1.076 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1974gitlab # [ 477.160138] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.025Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"8913dd35b497e7f969cfcb69","created_at":"2026-09-06T16:38:41.434Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.443Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8913dd35b497e7f969cfcb69: done: 0.068886 sec","job_status":"done","queue_duration_s":3.512894,"scheduling_latency_s":3.512922,"redis_calls":10,"redis_duration_s":0.002467,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00143,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028794,"duration_s":0.068886,"completed_at":"2026-09-06T16:38:45.025Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012354,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1975gitlab # [ 477.204167] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.027Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b1630ecd152cbc3298256a5d","created_at":"2026-09-06T16:38:41.457Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.465Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b1630ecd152cbc3298256a5d: start","job_status":"start","queue_duration_s":3.561295,"scheduling_latency_s":3.561313}1976gitlab # [ 477.220358] gitaly[893]: time="2026-09-06T16:38:45.092Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:55.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:45.083" grpc.time_ms=9.191 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1977gitlab # [ 477.233751] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.092Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b1630ecd152cbc3298256a5d","created_at":"2026-09-06T16:38:41.457Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.465Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b1630ecd152cbc3298256a5d: done: 0.065525 sec","job_status":"done","queue_duration_s":3.561295,"scheduling_latency_s":3.561313,"redis_calls":10,"redis_duration_s":0.004399,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000543,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003305,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00138302,"duration_s":0.065525,"completed_at":"2026-09-06T16:38:45.092Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016166,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1978gitlab # [ 477.277750] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.097Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"422f9b41cde75edff0bc1ec5","created_at":"2026-09-06T16:38:41.466Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.475Z","job_size_bytes":6,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-422f9b41cde75edff0bc1ec5: start","job_status":"start","queue_duration_s":3.621766,"scheduling_latency_s":3.621809}1979gitlab # [ 477.293381] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.147Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"422f9b41cde75edff0bc1ec5","created_at":"2026-09-06T16:38:41.466Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.475Z","job_size_bytes":6,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-422f9b41cde75edff0bc1ec5: done: 0.05058 sec","job_status":"done","queue_duration_s":3.621766,"scheduling_latency_s":3.621809,"redis_calls":10,"redis_duration_s":0.009814,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001444,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005249,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003121,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00335772,"duration_s":0.05058,"completed_at":"2026-09-06T16:38:45.147Z","load_balancing_strategy":"primary","db_duration_s":0.004503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1980gitlab # [ 477.338321] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.150Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"890c6e4a55a28d1676b7c9e8","created_at":"2026-09-06T16:38:41.476Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.481Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-890c6e4a55a28d1676b7c9e8: start","job_status":"start","queue_duration_s":3.669332,"scheduling_latency_s":3.669375}1981gitlab # [ 477.353728] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 3111982gitlab # [ 477.356290] nixbot[3813]: {"ts": "2026-09-06T16:38:45.181Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 \"HTTP/1.1 201 Created\""}1983gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'1984gitlab # [ 477.361529] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.221Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"890c6e4a55a28d1676b7c9e8","created_at":"2026-09-06T16:38:41.476Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.481Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-890c6e4a55a28d1676b7c9e8: done: 0.071492 sec","job_status":"done","queue_duration_s":3.669332,"scheduling_latency_s":3.669375,"redis_calls":10,"redis_duration_s":0.003666,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000987,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000703,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052438,"duration_s":0.071492,"completed_at":"2026-09-06T16:38:45.221Z","load_balancing_strategy":"primary","db_duration_s":0.017756,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1985gitlab # [ 477.402109] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.223Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f8bf36c6d14f950bebbafec1","created_at":"2026-09-06T16:38:41.489Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.496Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f8bf36c6d14f950bebbafec1: start","job_status":"start","queue_duration_s":3.727427,"scheduling_latency_s":3.727446}1986gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.07 seconds)1987gitlab # [ 477.418723] gitaly[893]: time="2026-09-06T16:38:45.301Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCAP7NQ14K1S0GTGQGWY8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:15.199" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:45.300" grpc.time_ms=0.807 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1988gitlab # [ 477.432473] nixbot[3813]: INFO: ::ffff:127.0.0.1:42798 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK1989gitlab # [ 477.450185] gitaly[893]: time="2026-09-06T16:38:45.394Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCAP7NQ14K1S0GTGQGWY8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:55.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:45.384" grpc.time_ms=10.103 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 477.463807] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.395Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f8bf36c6d14f950bebbafec1","created_at":"2026-09-06T16:38:41.489Z","trace_propagation_headers":{"sentry-trace":"b354fd33fde8446491bb4b212c23790d-b65b221e594042a0","baggage":"sentry-trace_id=b354fd33fde8446491bb4b212c23790d,sentry-release=Unknown"},"correlation_id":"01M1VS9N7K00CZWZ08Z8R32H84","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.496Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f8bf36c6d14f950bebbafec1: done: 0.171515 sec","job_status":"done","queue_duration_s":3.727427,"scheduling_latency_s":3.727446,"redis_calls":14,"redis_duration_s":0.006371,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000509,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00332,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":13,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":13,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00118742,"duration_s":0.171515,"completed_at":"2026-09-06T16:38:45.394Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034461,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1991gitlab # [ 477.509440] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.397Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a4876ea6b514613411d511d7","created_at":"2026-09-06T16:38:41.702Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-03211541cfc643d8","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"eaaa5b41a0a858c077310294","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.706Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a4876ea6b514613411d511d7: start","job_status":"start","queue_duration_s":3.690545,"scheduling_latency_s":3.690575}1992gitlab # [ 477.526318] gitaly[893]: time="2026-09-06T16:38:45.448Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX diskcache=a33cf44f-f19a-4391-93c3-fc3818482a7b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:38:45.435" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:45.435" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1993gitlab # [ 477.537483] gitaly[893]: time="2026-09-06T16:38:45.448Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-06T22:38:45.435" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:45.435" grpc.time_ms=12.276 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1994gitlab # [ 477.550578] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2881995gitlab # [ 477.553488] nixbot[3813]: {"ts": "2026-09-06T16:38:45.482Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 \"HTTP/1.1 201 Created\""}1996gitlab # [ 477.556255] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.480Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"a4876ea6b514613411d511d7","created_at":"2026-09-06T16:38:41.702Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-03211541cfc643d8","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"eaaa5b41a0a858c077310294","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.706Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-a4876ea6b514613411d511d7: done: 0.083136 sec","job_status":"done","queue_duration_s":3.690545,"scheduling_latency_s":3.690575,"gitaly_calls":1,"gitaly_duration_s":0.015746,"redis_calls":12,"redis_duration_s":0.0036690000000000004,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001396,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000776,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001497,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.014,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0465838,"duration_s":0.083136,"completed_at":"2026-09-06T16:38:45.480Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016051,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1997gitlab # [ 477.601838] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:45.482Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0f67eaf106d964573d16bac6","created_at":"2026-09-06T16:38:41.907Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-03211541cfc643d8","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4aec53ceb45322837c8c90452fbefbbbb8cbdfc2c3f7cb27cb5e155a94f10c96","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.913Z","job_size_bytes":467,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0f67eaf106d964573d16bac6: start","job_status":"start","queue_duration_s":3.569442,"scheduling_latency_s":3.569467}1998gitlab # [ 477.619214] gitaly[893]: time="2026-09-06T16:38:45.516Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:15.214" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:45.515" grpc.time_ms=0.756 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999gitlab # [ 477.631808] gitaly[893]: time="2026-09-06T16:38:45.536Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=6 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:38:55.089" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:45.519" grpc.time_ms=17.23 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2000gitlab # [ 477.644933] gitaly[893]: time="2026-09-06T16:38:45.586Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:39:15.285" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:45.585" grpc.time_ms=0.805 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2001gitlab # [ 477.657558] gitaly[893]: time="2026-09-06T16:38:45.595Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=49 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-06T16:39:15.291" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:45.591" grpc.time_ms=3.443 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2002gitlab # [ 477.673483] gitaly[893]: time="2026-09-06T16:38:45.598Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCAZKN8ZW662AF50CXHFZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:15.497" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:45.597" grpc.time_ms=0.697 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2003gitlab # [ 477.711469] gitaly[893]: time="2026-09-06T16:38:45.655Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:55.210" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:45.640" grpc.time_ms=15.397 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2004gitlab # [ 477.725096] gitaly[893]: time="2026-09-06T16:38:45.666Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCAZKN8ZW662AF50CXHFZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:55.276" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:45.657" grpc.time_ms=9.092 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2005gitlab # [ 477.795124] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 2472006gitlab # [ 477.799536] nixbot[3813]: {"ts": "2026-09-06T16:38:45.740Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/8e238942deb4adcaba3415f1c90e6142f3fd1b72 \"HTTP/1.1 201 Created\""}2007gitlab # [ 477.940371] gitaly[893]: time="2026-09-06T16:38:45.884Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCB8VRAYVFZ1T3QE9TMN3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:45.869" grpc.time_ms=14.348 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2008gitlab # [ 477.955868] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 972009gitlab # [ 478.305442] gitaly[893]: time="2026-09-06T16:38:46.249Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCBBZQREY858ABBVQWEPR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:46.229" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2010gitlab # [ 478.335496] gitaly[893]: time="2026-09-06T16:38:46.249Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCBBZQREY858ABBVQWEPR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:46.229" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2011gitlab # [ 478.367610] gitlab-workhorse[823]: time="2026-09-06T16:38:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672012gitlab # [ 478.371999] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 3632013gitlab # [ 478.376215] gitaly[893]: time="2026-09-06T16:38:46.250Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCBBZQREY858ABBVQWEPR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:46.229" grpc.time_ms=21.169 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2014gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'2015gitlab # [ 478.496971] nixbot[3813]: INFO: ::ffff:127.0.0.1:42814 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK2016gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.07 seconds)2017gitlab # [ 478.527925] gitaly[893]: time="2026-09-06T16:38:46.472Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=16 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-06T16:39:16.145" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:46.445" grpc.time_ms=25.974 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2018gitlab # [ 478.556885] gitaly[893]: time="2026-09-06T16:38:46.500Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSC19CVA3HN5NCJ65FAWPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-06T16:39:16.180" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:46.481" grpc.time_ms=19.389 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2019gitlab # [ 478.590359] sidekiq-cluster[3677]: {"severity":"WARN","time":"2026-09-06T16:38:46.533Z","class":"CreatePipelineWorker","correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"8e238942deb4adcaba3415f1c90e6142f3fd1b72","after":"4061fd525383d96028769d7d55bafd8ab2bd0ace","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4061fd525383d96028769d7d55bafd8ab2bd0ace","ref":"refs/heads/master"},"retry":0}2020gitlab # [ 478.604355] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.547Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"0f67eaf106d964573d16bac6","created_at":"2026-09-06T16:38:41.907Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-03211541cfc643d8","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4aec53ceb45322837c8c90452fbefbbbb8cbdfc2c3f7cb27cb5e155a94f10c96","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.913Z","job_size_bytes":467,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-0f67eaf106d964573d16bac6: done: 1.064726 sec","job_status":"done","queue_duration_s":3.569442,"scheduling_latency_s":3.569467,"gitaly_calls":7,"gitaly_duration_s":0.110603,"redis_calls":33,"redis_duration_s":0.0072640000000000005,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001087,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002824,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001645,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.926126,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.064726,"completed_at":"2026-09-06T16:38:46.547Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030854,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2021gitlab # [ 478.675043] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.551Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"34636628-6612-4516-a212-a067487b2372\"}"],"class":"WebHookWorker","jid":"d3c170a33d56a92b10507976","created_at":"2026-09-06T16:38:41.950Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-03211541cfc643d8","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:813b7e801d906de1d98095a8689338a1174000dd6b85e07f814ae7fc7168789d","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.955Z","job_size_bytes":1749,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d3c170a33d56a92b10507976: start","job_status":"start","queue_duration_s":4.595754,"scheduling_latency_s":4.595806}2022gitlab # [ 478.697506] nixbot[3813]: INFO: ::ffff:127.0.0.1:42828 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2023gitlab # [ 478.699131] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.632Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"34636628-6612-4516-a212-a067487b2372\"}"],"class":"WebHookWorker","jid":"d3c170a33d56a92b10507976","created_at":"2026-09-06T16:38:41.950Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-03211541cfc643d8","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:813b7e801d906de1d98095a8689338a1174000dd6b85e07f814ae7fc7168789d","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:41.955Z","job_size_bytes":1749,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-d3c170a33d56a92b10507976: done: 0.081372 sec","job_status":"done","queue_duration_s":4.595754,"scheduling_latency_s":4.595806,"redis_calls":14,"redis_duration_s":0.005615,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000761,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003016,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001838,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.01225640000001249,"cpu_s":0.052116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.081372,"completed_at":"2026-09-06T16:38:46.632Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004295,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2024gitlab # [ 478.748492] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.635Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"12a3f4f8eb8e1bacbfce65c7","created_at":"2026-09-06T16:38:42.187Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-6e5f1c29fcce4863","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:42.195Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-12a3f4f8eb8e1bacbfce65c7: start","job_status":"start","queue_duration_s":4.44059,"scheduling_latency_s":4.440632}2025gitlab # [ 478.790576] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.734Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"12a3f4f8eb8e1bacbfce65c7","created_at":"2026-09-06T16:38:42.187Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-6e5f1c29fcce4863","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:42.195Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-12a3f4f8eb8e1bacbfce65c7: done: 0.098659 sec","job_status":"done","queue_duration_s":4.44059,"scheduling_latency_s":4.440632,"redis_calls":19,"redis_duration_s":0.008064,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00164,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000877,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003547,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.002,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.070904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01649558,"duration_s":0.098659,"completed_at":"2026-09-06T16:38:46.734Z","load_balancing_strategy":"primary","db_duration_s":0.014886,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2026gitlab # [ 478.840586] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.736Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","46fb3e19b9873e6d1bc76f54"],"class":"WebHooks::LogExecutionWorker","jid":"4b55c8c19558dac95dd16205","created_at":"2026-09-06T16:38:44.237Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-919244478005443c","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1ec50272f4c11740091ea30618ed56bab611f489d3cc07c9fe653ce71c37e0b2","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:44.253Z","job_size_bytes":2593,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4b55c8c19558dac95dd16205: start","job_status":"start","queue_duration_s":2.48344,"scheduling_latency_s":2.483478}2027gitlab # [ 478.857457] gitaly[893]: time="2026-09-06T16:38:46.800Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=6 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCC4567JP7DG9KX1F9J23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:46.780" grpc.time_ms=19.573 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2028gitlab # [ 478.871639] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:46 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1382029gitlab # [ 478.880632] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.824Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","46fb3e19b9873e6d1bc76f54"],"class":"WebHooks::LogExecutionWorker","jid":"4b55c8c19558dac95dd16205","created_at":"2026-09-06T16:38:44.237Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-919244478005443c","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1ec50272f4c11740091ea30618ed56bab611f489d3cc07c9fe653ce71c37e0b2","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:44.253Z","job_size_bytes":2593,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-4b55c8c19558dac95dd16205: done: 0.087385 sec","job_status":"done","queue_duration_s":2.48344,"scheduling_latency_s":2.483478,"redis_calls":10,"redis_duration_s":0.0037949999999999998,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000726,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002299,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00077,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007699,"duration_s":0.087385,"completed_at":"2026-09-06T16:38:46.824Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014835,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2030gitlab # [ 478.925656] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:46.826Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e64105534d51951770106348","created_at":"2026-09-06T16:38:45.145Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-2815f34904c14dfb","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:45.153Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e64105534d51951770106348: start","job_status":"start","queue_duration_s":1.673744,"scheduling_latency_s":1.673782}2031gitlab # [ 478.974053] gitaly[893]: time="2026-09-06T16:38:46.917Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCC8MC1PD5CKX4V1AHNVC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:46.899" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2032gitlab # [ 478.985781] gitaly[893]: time="2026-09-06T16:38:46.917Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCC8MC1PD5CKX4V1AHNVC grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:46.899" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2033gitlab # [ 478.999621] gitlab-workhorse[823]: time="2026-09-06T16:38:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672034gitlab # [ 479.001611] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 1132035gitlab # [ 479.003525] gitaly[893]: time="2026-09-06T16:38:46.918Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCC8MC1PD5CKX4V1AHNVC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:46.899" grpc.time_ms=19.072 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2036gitlab # [ 479.056357] gitaly[893]: time="2026-09-06T16:38:47.000Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:46.999" grpc.time_ms=0.843 pid=893 span.kind=server system=grpc2037gitlab # [ 479.095709] gitaly[893]: time="2026-09-06T16:38:47.039Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3895 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:47.039" grpc.time_ms=0.587 pid=893 span.kind=server system=grpc2038gitlab # [ 479.142582] gitaly[893]: time="2026-09-06T16:38:47.086Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3897 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:47.085" grpc.time_ms=0.973 pid=893 span.kind=server system=grpc2039gitlab # [ 479.157387] gitaly[893]: time="2026-09-06T16:38:47.088Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCCE19H4SQNAY19AF8RG7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:47.072" grpc.time_ms=16.271 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2040gitlab # [ 479.176678] gitaly[893]: time="2026-09-06T16:38:47.100Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS diskcache=238646c4-4e13-4adb-b6b2-da08f58a5f5d grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:39:16.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:46.956" pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2041gitlab # [ 479.189708] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 1112042gitlab # [ 479.192402] gitaly[893]: time="2026-09-06T16:38:47.101Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=136 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7432 command.oublock=0 command.real_time_ms=135 command.spawn_token_fork_ms=4 command.system_time_ms=84 command.user_time_ms=52 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-06T16:39:16.655" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:46.956" grpc.time_ms=144.354 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2043gitlab # [ 479.256283] gitaly[893]: time="2026-09-06T16:38:47.200Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:39:17.198" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:47.198" grpc.time_ms=1.365 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2044gitlab # [ 479.307280] gitaly[893]: time="2026-09-06T16:38:47.251Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCHKXPTN8DD9P1N7SVG7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:47.211" pid=893 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2045gitlab # [ 479.329823] gitaly[893]: time="2026-09-06T16:38:47.251Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=25 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=5 command.system_time_ms=20 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCHKXPTN8DD9P1N7SVG7 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:47.211" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2046gitlab # [ 479.353364] gitlab-workhorse[823]: time="2026-09-06T16:38:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162047gitlab # [ 479.356356] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1612048gitlab # [ 479.359145] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.284Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e64105534d51951770106348","created_at":"2026-09-06T16:38:45.145Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-2815f34904c14dfb","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:45.153Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3677,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e64105534d51951770106348: rescheduled"}2049gitlab # [ 479.381620] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.297Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"e64105534d51951770106348","created_at":"2026-09-06T16:38:45.145Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-2815f34904c14dfb","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:45.153Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e64105534d51951770106348: done: 0.470797 sec","job_status":"done","queue_duration_s":1.673744,"scheduling_latency_s":1.673782,"gitaly_calls":3,"gitaly_duration_s":0.161549,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.033195,"redis_read_bytes":148,"redis_write_bytes":18011,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.003884,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002196,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.006704,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.010622,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000601,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001064,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00424,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.013,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.013,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.231508,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.39560414,"duration_s":0.470797,"completed_at":"2026-09-06T16:38:47.297Z","load_balancing_strategy":"primary","db_duration_s":0.055341,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2050gitlab # [ 479.444398] gitaly[893]: time="2026-09-06T16:38:47.252Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=25 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=5 command.system_time_ms=20 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCHKXPTN8DD9P1N7SVG7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:47.211" grpc.time_ms=41.007 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2051gitlab # [ 479.460464] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.300Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d3c170a33d56a92b10507976"],"class":"WebHooks::LogExecutionWorker","jid":"0a92ce573a754892b09146c9","created_at":"2026-09-06T16:38:46.612Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-6976c61929f14aae","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a20b0182fca36a5f8f024dd7a5798548c0310f39c0c057e535ee9fdae7af414","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:46.617Z","job_size_bytes":2592,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-0a92ce573a754892b09146c9: start","job_status":"start","queue_duration_s":0.6825,"scheduling_latency_s":0.68254}2052gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace'2053gitlab # [ 479.478439] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.340Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","d3c170a33d56a92b10507976"],"class":"WebHooks::LogExecutionWorker","jid":"0a92ce573a754892b09146c9","created_at":"2026-09-06T16:38:46.612Z","trace_propagation_headers":{"sentry-trace":"ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c-6976c61929f14aae","baggage":"sentry-trace_id=ef5e7c4ed5cf4b91bffc5bb4fe3a1c6c,sentry-release=Unknown"},"correlation_id":"01M1VSC19CVA3HN5NCJ65FAWPX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7a20b0182fca36a5f8f024dd7a5798548c0310f39c0c057e535ee9fdae7af414","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:46.617Z","job_size_bytes":2592,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-0a92ce573a754892b09146c9: done: 0.04053 sec","job_status":"done","queue_duration_s":0.6825,"scheduling_latency_s":0.68254,"redis_calls":7,"redis_duration_s":0.002167,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001581,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028442,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04053,"completed_at":"2026-09-06T16:38:47.340Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004486,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2054gitlab # [ 479.522378] gitaly[893]: time="2026-09-06T16:38:47.265Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCAC3RR0PY10029K6ETFS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:17.263" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:47.264" grpc.time_ms=1.267 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2055gitlab # [ 479.535616] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"319747df5a84fb99dd134c3c","created_at":"2026-09-06T16:38:46.707Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-4148fe78ac914caa","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:46.719Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-319747df5a84fb99dd134c3c: start","job_status":"start","queue_duration_s":0.623708,"scheduling_latency_s":0.623744}2056gitlab # [ 479.553786] gitaly[893]: time="2026-09-06T16:38:47.431Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:17.327" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:47.428" grpc.time_ms=2.779 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2057gitlab # [ 479.566890] gitaly[893]: time="2026-09-06T16:38:47.471Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=4 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-06T16:39:17.348" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:47.449" grpc.time_ms=21.905 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2058gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace', in 0.09 seconds)2059(finished: subtest: push delivers the webhook, in 91.32 seconds)2060subtest: eval and build post commit statuses2061gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2062gitlab # [ 479.580936] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.510Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"319747df5a84fb99dd134c3c","created_at":"2026-09-06T16:38:46.707Z","trace_propagation_headers":{"sentry-trace":"6cefb6ab50a54e70b15f707b1855d651-4148fe78ac914caa","baggage":"sentry-trace_id=6cefb6ab50a54e70b15f707b1855d651,sentry-release=Unknown"},"correlation_id":"01M1VS9E6B5EHZBCHH9HP0K518","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:46.719Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-319747df5a84fb99dd134c3c: done: 0.167254 sec","job_status":"done","queue_duration_s":0.623708,"scheduling_latency_s":0.623744,"redis_calls":11,"redis_duration_s":0.00611,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000931,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003765,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001414,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.014,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.111711,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00156398,"duration_s":0.167254,"completed_at":"2026-09-06T16:38:47.509Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2063gitlab # [ 479.631914] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.513Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b7ddf5c08c044314ec780da2","created_at":"2026-09-06T16:38:46.943Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:46.946Z","job_size_bytes":23,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b7ddf5c08c044314ec780da2: start","job_status":"start","queue_duration_s":0.56594,"scheduling_latency_s":0.565985}2064gitlab # [ 479.648687] gitaly[893]: time="2026-09-06T16:38:47.503Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=8 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:38:57.108" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:47.489" grpc.time_ms=13.867 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2065gitlab # [ 479.662294] nixbot[3813]: INFO: ::ffff:127.0.0.1:42842 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 200 OK2066gitlab # [ 479.664279] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.602Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"b7ddf5c08c044314ec780da2","created_at":"2026-09-06T16:38:46.943Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:46.946Z","job_size_bytes":23,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-b7ddf5c08c044314ec780da2: done: 0.090109 sec","job_status":"done","queue_duration_s":0.56594,"scheduling_latency_s":0.565985,"redis_calls":7,"redis_duration_s":0.002158,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001507,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000651,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062892,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090109,"completed_at":"2026-09-06T16:38:47.602Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018873,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2067gitlab # [ 479.708370] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.605Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f3b8d01b3c4f95e9dd812934","created_at":"2026-09-06T16:38:47.112Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.117Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f3b8d01b3c4f95e9dd812934: start","job_status":"start","queue_duration_s":0.487529,"scheduling_latency_s":0.487565}2068gitlab # [ 479.725401] gitaly[893]: time="2026-09-06T16:38:47.559Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-06T16:38:57.178" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:47.558" grpc.time_ms=0.392 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2069gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.19 seconds)2070gitlab # [ 479.736617] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.654Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"f3b8d01b3c4f95e9dd812934","created_at":"2026-09-06T16:38:47.112Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.117Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-f3b8d01b3c4f95e9dd812934: done: 0.049728 sec","job_status":"done","queue_duration_s":0.487529,"scheduling_latency_s":0.487565,"redis_calls":7,"redis_duration_s":0.001817,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001296,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036716,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049728,"completed_at":"2026-09-06T16:38:47.654Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2071gitlab # [ 479.780512] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.656Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"09d1abac36891aa98da571ff","created_at":"2026-09-06T16:38:47.131Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.136Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-09d1abac36891aa98da571ff: start","job_status":"start","queue_duration_s":0.520101,"scheduling_latency_s":0.520137}2072gitlab # [ 479.797759] gitaly[893]: time="2026-09-06T16:38:47.577Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-06T16:38:57.186" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:47.567" grpc.time_ms=9.754 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2073gitlab # [ 479.811384] gitaly[893]: time="2026-09-06T16:38:47.629Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4153 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:47.628" grpc.time_ms=0.626 pid=893 span.kind=server system=grpc2074gitlab # [ 479.821465] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:47 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 1492075gitlab # [ 479.824137] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.717Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"09d1abac36891aa98da571ff","created_at":"2026-09-06T16:38:47.131Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.136Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-09d1abac36891aa98da571ff: done: 0.061094 sec","job_status":"done","queue_duration_s":0.520101,"scheduling_latency_s":0.520137,"redis_calls":7,"redis_duration_s":0.002058,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001472,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000586,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.061094,"completed_at":"2026-09-06T16:38:47.717Z","load_balancing_strategy":"primary","db_duration_s":0.011615,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 479.866923] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.720Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"77aed0b79cd86a08ece170a5","created_at":"2026-09-06T16:38:47.150Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.154Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-77aed0b79cd86a08ece170a5: start","job_status":"start","queue_duration_s":0.566312,"scheduling_latency_s":0.566353}2077gitlab # [ 479.882971] gitaly[893]: time="2026-09-06T16:38:47.675Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCD0DFYH4T58TCNMERN6N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:17.672" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:47.673" grpc.time_ms=1.98 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078gitlab # [ 479.896051] gitaly[893]: time="2026-09-06T16:38:47.681Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-06T16:38:47.675" grpc.time_ms=5.351 pid=893 span.kind=server system=grpc2079gitlab # [ 479.905912] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.777Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"77aed0b79cd86a08ece170a5","created_at":"2026-09-06T16:38:47.150Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.154Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-77aed0b79cd86a08ece170a5: done: 0.057744 sec","job_status":"done","queue_duration_s":0.566312,"scheduling_latency_s":0.566353,"redis_calls":10,"redis_duration_s":0.0024920000000000003,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000701,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057744,"completed_at":"2026-09-06T16:38:47.777Z","load_balancing_strategy":"primary","db_duration_s":0.009862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2080gitlab # [ 479.951643] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.780Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fe38fdec12f473d0ea0637ac","created_at":"2026-09-06T16:38:47.154Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.158Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fe38fdec12f473d0ea0637ac: start","job_status":"start","queue_duration_s":0.621856,"scheduling_latency_s":0.621893}2081gitlab # [ 479.968811] gitaly[893]: time="2026-09-06T16:38:47.696Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ diskcache=60e8248c-ea6b-40d4-9f8b-4d823a021942 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:38:57.201" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:47.581" pid=893 ref=refs/keep-around/4061fd525383d96028769d7d55bafd8ab2bd0ace remote_ip="2001:db8:1::1" revision=4061fd525383d96028769d7d55bafd8ab2bd0ace span.kind=server system=grpc username=root2082gitlab # [ 479.980996] nixbot[3813]: {"ts": "2026-09-06T16:38:47.858Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace \"HTTP/1.1 201 Created\""}2083gitlab # [ 479.983835] nixbot[3813]: {"ts": "2026-09-06T16:38:47.875Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-4061fd525383-5"}2084gitlab # [ 479.986381] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.828Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fe38fdec12f473d0ea0637ac","created_at":"2026-09-06T16:38:47.154Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.158Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fe38fdec12f473d0ea0637ac: done: 0.048358 sec","job_status":"done","queue_duration_s":0.621856,"scheduling_latency_s":0.621893,"redis_calls":7,"redis_duration_s":0.001972,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001436,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000536,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048358,"completed_at":"2026-09-06T16:38:47.828Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007312,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2085gitlab # [ 480.031389] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.833Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"13485c66b702419634aeb0f0","created_at":"2026-09-06T16:38:47.159Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.164Z","job_size_bytes":19,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-13485c66b702419634aeb0f0: start","job_status":"start","queue_duration_s":0.669233,"scheduling_latency_s":0.669273}2086gitlab # [ 480.048743] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:47 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 5492087gitlab # [ 480.051666] gitaly[893]: time="2026-09-06T16:38:47.697Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=85 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5280 command.oublock=8 command.real_time_ms=104 command.spawn_token_fork_ms=9 command.system_time_ms=23 command.user_time_ms=62 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-06T16:38:57.201" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-06T16:38:47.581" grpc.time_ms=115.135 pid=893 ref=refs/keep-around/4061fd525383d96028769d7d55bafd8ab2bd0ace remote_ip="2001:db8:1::1" revision=4061fd525383d96028769d7d55bafd8ab2bd0ace span.kind=server system=grpc username=root2088gitlab # [ 480.066694] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.897Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"13485c66b702419634aeb0f0","created_at":"2026-09-06T16:38:47.159Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.164Z","job_size_bytes":19,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-13485c66b702419634aeb0f0: done: 0.063654 sec","job_status":"done","queue_duration_s":0.669233,"scheduling_latency_s":0.669273,"redis_calls":8,"redis_duration_s":0.0034530000000000003,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002638,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063654,"completed_at":"2026-09-06T16:38:47.897Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2089gitlab # [ 480.109517] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.901Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a8634d6c0aff1f951c56723e","created_at":"2026-09-06T16:38:47.166Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.169Z","job_size_bytes":66,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a8634d6c0aff1f951c56723e: start","job_status":"start","queue_duration_s":0.731559,"scheduling_latency_s":0.73161}2090gitlab # [ 480.126678] gitaly[893]: time="2026-09-06T16:38:47.702Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:17.600" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:47.701" grpc.time_ms=0.802 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2091gitlab # [ 480.139403] nixbot[3813]: {"ts": "2026-09-06T16:38:48.024Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4061fd525383-5", "argv0": "bwrap"}2092gitlab # [ 480.141912] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.970Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a8634d6c0aff1f951c56723e","created_at":"2026-09-06T16:38:47.166Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.169Z","job_size_bytes":66,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a8634d6c0aff1f951c56723e: done: 0.068669 sec","job_status":"done","queue_duration_s":0.731559,"scheduling_latency_s":0.73161,"redis_calls":7,"redis_duration_s":0.004814000000000001,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.068669,"completed_at":"2026-09-06T16:38:47.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2093gitlab # [ 480.185433] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:47.975Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"670d08191210c8240b047bef","created_at":"2026-09-06T16:38:47.170Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.174Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-670d08191210c8240b047bef: start","job_status":"start","queue_duration_s":0.801718,"scheduling_latency_s":0.80178}2094gitlab # [ 480.201832] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.042Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"670d08191210c8240b047bef","created_at":"2026-09-06T16:38:47.170Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.174Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-670d08191210c8240b047bef: done: 0.066777 sec","job_status":"done","queue_duration_s":0.801718,"scheduling_latency_s":0.80178,"redis_calls":7,"redis_duration_s":0.0034590000000000003,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002292,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001167,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.066777,"completed_at":"2026-09-06T16:38:48.042Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00952,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2095gitlab # [ 480.245702] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.044Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1d47a3be366e11072940eaa3","created_at":"2026-09-06T16:38:47.174Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.177Z","job_size_bytes":6,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1d47a3be366e11072940eaa3: start","job_status":"start","queue_duration_s":0.866958,"scheduling_latency_s":0.866996}2096gitlab # [ 480.262052] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.069Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1d47a3be366e11072940eaa3","created_at":"2026-09-06T16:38:47.174Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.177Z","job_size_bytes":6,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1d47a3be366e11072940eaa3: done: 0.024983 sec","job_status":"done","queue_duration_s":0.866958,"scheduling_latency_s":0.866996,"redis_calls":7,"redis_duration_s":0.002404,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001734,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00067,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024983,"completed_at":"2026-09-06T16:38:48.069Z","load_balancing_strategy":"primary","db_duration_s":0.000914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2097gitlab # [ 480.304233] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.072Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"45a4c38842a6f74f785299b7","created_at":"2026-09-06T16:38:47.178Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.181Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-45a4c38842a6f74f785299b7: start","job_status":"start","queue_duration_s":0.890416,"scheduling_latency_s":0.890449}2098gitlab # [ 480.319620] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"45a4c38842a6f74f785299b7","created_at":"2026-09-06T16:38:47.178Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.181Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-45a4c38842a6f74f785299b7: done: 0.043975 sec","job_status":"done","queue_duration_s":0.890416,"scheduling_latency_s":0.890449,"redis_calls":7,"redis_duration_s":0.002502,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001817,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000685,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029798,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043975,"completed_at":"2026-09-06T16:38:48.115Z","load_balancing_strategy":"primary","db_duration_s":0.00729,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2099gitlab # [ 480.361288] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.118Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a998f452663c4538dd4b08fb","created_at":"2026-09-06T16:38:47.285Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.289Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a998f452663c4538dd4b08fb: start","job_status":"start","queue_duration_s":0.829543,"scheduling_latency_s":0.829583}2100gitlab # [ 480.377357] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.167Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"a998f452663c4538dd4b08fb","created_at":"2026-09-06T16:38:47.285Z","trace_propagation_headers":{"sentry-trace":"d98f8fb9a3554159b95b70bd138d877e-cb6cdbd2718f4d61","baggage":"sentry-trace_id=d98f8fb9a3554159b95b70bd138d877e,sentry-release=Unknown"},"correlation_id":"01M1VSCAC3RR0PY10029K6ETFS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.289Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-a998f452663c4538dd4b08fb: done: 0.0492 sec","job_status":"done","queue_duration_s":0.829543,"scheduling_latency_s":0.829583,"redis_calls":8,"redis_duration_s":0.002892,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002262,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00063,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0492,"completed_at":"2026-09-06T16:38:48.167Z","load_balancing_strategy":"primary","db_duration_s":0.008268,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2101gitlab # [ 480.420160] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.170Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ddd61dafa3a7cfc33d82d11f","created_at":"2026-09-06T16:38:47.713Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-24d968986adc4043","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.719Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ddd61dafa3a7cfc33d82d11f: start","job_status":"start","queue_duration_s":0.450967,"scheduling_latency_s":0.451004}2102gitlab # [ 480.437129] nix-daemon[3985]: accepted connection from pid 5033, user nixbot2103gitlab # [ 480.438160] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.260Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ddd61dafa3a7cfc33d82d11f","created_at":"2026-09-06T16:38:47.713Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-24d968986adc4043","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.719Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ddd61dafa3a7cfc33d82d11f: done: 0.090391 sec","job_status":"done","queue_duration_s":0.450967,"scheduling_latency_s":0.451004,"redis_calls":8,"redis_duration_s":0.030286999999999998,"redis_read_bytes":39,"redis_write_bytes":766,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000403,"redis_feature_flag_read_bytes":34,"redis_feature_flag_write_bytes":35,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.017608,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.012276,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044714,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.090391,"completed_at":"2026-09-06T16:38:48.260Z","load_balancing_strategy":"primary","db_duration_s":0.010821,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2104gitlab # [ 480.482753] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.263Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d6eb4961c23b89e26998d207","created_at":"2026-09-06T16:38:47.720Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-24d968986adc4043","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.727Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d6eb4961c23b89e26998d207: start","job_status":"start","queue_duration_s":0.536059,"scheduling_latency_s":0.536096}2105gitlab # [ 480.500465] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.419Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"d6eb4961c23b89e26998d207","created_at":"2026-09-06T16:38:47.720Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-24d968986adc4043","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.727Z","job_size_bytes":65,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-d6eb4961c23b89e26998d207: done: 0.156829 sec","job_status":"done","queue_duration_s":0.536059,"scheduling_latency_s":0.536096,"redis_calls":10,"redis_duration_s":0.006428,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003471,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.002957,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.105855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.156829,"completed_at":"2026-09-06T16:38:48.419Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040062,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106gitlab # [ 480.544389] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.423Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7e6e51eff9d47fe811cb6113","created_at":"2026-09-06T16:38:47.816Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-24d968986adc4043","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.822Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7e6e51eff9d47fe811cb6113: start","job_status":"start","queue_duration_s":0.600634,"scheduling_latency_s":0.600684}2107gitlab # [ 480.644833] gitaly[893]: time="2026-09-06T16:38:48.589Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:39:18.286" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:48.586" grpc.time_ms=1.72 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2108gitlab # [ 480.743047] gitaly[893]: time="2026-09-06T16:38:48.687Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCCRBAHZYDZ8MPGGBK2YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:18.384" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:48.685" grpc.time_ms=1.397 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2109gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2110gitlab # [ 480.781142] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.712Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7e6e51eff9d47fe811cb6113","created_at":"2026-09-06T16:38:47.816Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-24d968986adc4043","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:47.822Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7e6e51eff9d47fe811cb6113: done: 0.288667 sec","job_status":"done","queue_duration_s":0.600634,"scheduling_latency_s":0.600684,"gitaly_calls":2,"gitaly_duration_s":0.013119,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.014169,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002349,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001892,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000588,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003134,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000565,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000932,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00236,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.05,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.024,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.224928,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.21596624,"duration_s":0.288667,"completed_at":"2026-09-06T16:38:48.711Z","load_balancing_strategy":"primary","db_duration_s":0.054267,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2111gitlab # [ 480.866799] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.715Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"2307e1535a2b63353833274a","created_at":"2026-09-06T16:38:48.551Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-bd904f56afa84922","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:48.556Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2307e1535a2b63353833274a: start","job_status":"start","queue_duration_s":0.158815,"scheduling_latency_s":0.158902}2112gitlab # [ 480.999434] gitaly[893]: time="2026-09-06T16:38:48.943Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCE7C8JHTCWCMEYW6S585 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:18.841" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:48.942" grpc.time_ms=1.219 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2113gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.30 seconds)2114gitlab # [ 481.039688] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:48.977Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"2307e1535a2b63353833274a","created_at":"2026-09-06T16:38:48.551Z","trace_propagation_headers":{"sentry-trace":"a522f64aa3714cab9d6c692f0691ca13-bd904f56afa84922","baggage":"sentry-trace_id=a522f64aa3714cab9d6c692f0691ca13,sentry-release=Unknown"},"correlation_id":"01M1VSCCRBAHZYDZ8MPGGBK2YJ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:48.556Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2307e1535a2b63353833274a: done: 0.261623 sec","job_status":"done","queue_duration_s":0.158815,"scheduling_latency_s":0.158902,"redis_calls":7,"redis_duration_s":0.00331,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000889,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.235136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.261623,"completed_at":"2026-09-06T16:38:48.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011695,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2115gitlab # [ 481.121749] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 2042116gitlab # [ 481.547495] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:49.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"177812305616147ae2912e72","created_at":"2026-09-06T16:38:49.460Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41c4052fcf2a2779790c3ddcfc0aaf24","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.482Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-177812305616147ae2912e72: start","job_status":"start","queue_duration_s":0.006223,"scheduling_latency_s":0.006286}2117gitlab # [ 481.674206] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:49.618Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2118gitlab # [ 481.708342] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:49.651Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"177812305616147ae2912e72","created_at":"2026-09-06T16:38:49.460Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41c4052fcf2a2779790c3ddcfc0aaf24","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.482Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-177812305616147ae2912e72: done: 0.162961 sec","job_status":"done","queue_duration_s":0.006223,"scheduling_latency_s":0.006286,"redis_calls":9,"redis_duration_s":0.019742,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003196,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004021,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042886,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.162961,"completed_at":"2026-09-06T16:38:49.651Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009865,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119gitlab # [ 481.801244] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:49.655Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"19035aae26993c05d0116c8c","created_at":"2026-09-06T16:38:49.522Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c0d5b84f5eac8404183c5e92b13bab9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.540Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-19035aae26993c05d0116c8c: start","job_status":"start","queue_duration_s":0.114901,"scheduling_latency_s":0.114954}2120gitlab # [ 481.868631] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:49.810Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"19035aae26993c05d0116c8c","created_at":"2026-09-06T16:38:49.522Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7c0d5b84f5eac8404183c5e92b13bab9","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.540Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-19035aae26993c05d0116c8c: done: 0.154854 sec","job_status":"done","queue_duration_s":0.114901,"scheduling_latency_s":0.114954,"redis_calls":11,"redis_duration_s":0.019927,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00439,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006373,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009164,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00205552,"exclusive_lock_hold_duration_s":0.02999362,"duration_s":0.154854,"completed_at":"2026-09-06T16:38:49.809Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 481.964084] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:49.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a4cf55478e1a4821dab84b30","created_at":"2026-09-06T16:38:49.559Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0d6228223a4f75c85fef46ec21662ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:38:49.589Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a4cf55478e1a4821dab84b30: start","job_status":"start","queue_duration_s":0.235893,"scheduling_latency_s":0.235949}2122gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2123gitlab # [ 482.158772] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"a4cf55478e1a4821dab84b30","created_at":"2026-09-06T16:38:49.559Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0d6228223a4f75c85fef46ec21662ec","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-06T16:38:49.589Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-a4cf55478e1a4821dab84b30: done: 0.275506 sec","job_status":"done","queue_duration_s":0.235893,"scheduling_latency_s":0.235949,"redis_calls":9,"redis_duration_s":0.025033,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002758,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.014968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007307,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.063,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.275506,"completed_at":"2026-09-06T16:38:50.101Z","load_balancing_strategy":"primary","db_duration_s":0.072286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2124gitlab # [ 482.243697] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.108Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6f8e9e24fbcd992e9ad03658","created_at":"2026-09-06T16:38:49.615Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.706Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6f8e9e24fbcd992e9ad03658: start","job_status":"start","queue_duration_s":0.40251,"scheduling_latency_s":0.402568}2125gitlab # [ 482.260767] gitaly[893]: time="2026-09-06T16:38:50.179Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCFF46ZER4XWMTDMGJJGT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:20.177" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:50.178" grpc.time_ms=0.845 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2126gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.20 seconds)2127gitlab # [ 482.276770] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1332128gitlab # [ 482.439697] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"6f8e9e24fbcd992e9ad03658","created_at":"2026-09-06T16:38:49.615Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.706Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-6f8e9e24fbcd992e9ad03658: done: 0.273988 sec","job_status":"done","queue_duration_s":0.40251,"scheduling_latency_s":0.402568,"redis_calls":18,"redis_duration_s":0.082286,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.014806,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.003038,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.03837,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.026072,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.084557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.273988,"completed_at":"2026-09-06T16:38:50.382Z","load_balancing_strategy":"primary","db_duration_s":0.017153,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2129gitlab # [ 482.541565] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.425Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fa91afa4f593dc3df6697347","created_at":"2026-09-06T16:38:49.719Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc1cff7df3e383f8d0148cae8fee8f50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:38:49.744Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fa91afa4f593dc3df6697347: start","job_status":"start","queue_duration_s":0.680365,"scheduling_latency_s":0.680427}2130gitlab # [ 482.577902] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.521Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2131gitlab # [ 482.629150] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.572Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fa91afa4f593dc3df6697347","created_at":"2026-09-06T16:38:49.719Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dc1cff7df3e383f8d0148cae8fee8f50","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-06T16:38:49.744Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fa91afa4f593dc3df6697347: done: 0.14707 sec","job_status":"done","queue_duration_s":0.680365,"scheduling_latency_s":0.680427,"redis_calls":9,"redis_duration_s":0.020072,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003982,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008323,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.007767,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.14707,"completed_at":"2026-09-06T16:38:50.572Z","load_balancing_strategy":"primary","db_duration_s":0.014626,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2132gitlab # [ 482.728387] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.588Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6ffb6dc11cfc3b98198f0f55","created_at":"2026-09-06T16:38:49.800Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0159da39a23a9d5c52f403678aa3a59c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.822Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6ffb6dc11cfc3b98198f0f55: start","job_status":"start","queue_duration_s":0.765272,"scheduling_latency_s":0.765331}2133gitlab # [ 482.864619] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.806Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"6ffb6dc11cfc3b98198f0f55","created_at":"2026-09-06T16:38:49.800Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0159da39a23a9d5c52f403678aa3a59c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.822Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-6ffb6dc11cfc3b98198f0f55: done: 0.218498 sec","job_status":"done","queue_duration_s":0.765272,"scheduling_latency_s":0.765331,"redis_calls":11,"redis_duration_s":0.020574000000000002,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.006792,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.010642,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.218498,"completed_at":"2026-09-06T16:38:50.806Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021685,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134gitlab # [ 482.961548] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.818Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fc9942067c5b35eb25c022ee","created_at":"2026-09-06T16:38:49.839Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00ff69cff13fd54a0a1815aa9ee47e81","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.854Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fc9942067c5b35eb25c022ee: start","job_status":"start","queue_duration_s":0.963849,"scheduling_latency_s":0.963907}2135gitlab # [ 483.002193] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"fc9942067c5b35eb25c022ee","created_at":"2026-09-06T16:38:49.839Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00ff69cff13fd54a0a1815aa9ee47e81","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.854Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-fc9942067c5b35eb25c022ee: done: 0.126842 sec","job_status":"done","queue_duration_s":0.963849,"scheduling_latency_s":0.963907,"redis_calls":9,"redis_duration_s":0.013522,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001687,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008357,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003478,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.126842,"completed_at":"2026-09-06T16:38:50.945Z","load_balancing_strategy":"primary","db_duration_s":0.032863,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2136gitlab # [ 483.071033] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.949Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7d42c061b34eea6eba8a3c1b","created_at":"2026-09-06T16:38:49.865Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"257ae548735b7bd33fe862da41f47ab6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.896Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7d42c061b34eea6eba8a3c1b: start","job_status":"start","queue_duration_s":1.053036,"scheduling_latency_s":1.053092}2137gitlab # [ 483.088846] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.980Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2138gitlab # [ 483.092867] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.996Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"7d42c061b34eea6eba8a3c1b","created_at":"2026-09-06T16:38:49.865Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"257ae548735b7bd33fe862da41f47ab6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:49.896Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-7d42c061b34eea6eba8a3c1b: done: 0.047251 sec","job_status":"done","queue_duration_s":1.053036,"scheduling_latency_s":1.053092,"redis_calls":9,"redis_duration_s":0.003653,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000804,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002053,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000796,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047251,"completed_at":"2026-09-06T16:38:50.996Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2139gitlab # [ 483.139537] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:50.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"96eda23b888c44fb57ff2a93","created_at":"2026-09-06T16:38:49.974Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7fbd15dbede12ecfc1a3074bd9f2719","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.025Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-96eda23b888c44fb57ff2a93: start","job_status":"start","queue_duration_s":0.974313,"scheduling_latency_s":0.974356}2140gitlab # [ 483.153201] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"96eda23b888c44fb57ff2a93","created_at":"2026-09-06T16:38:49.974Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7fbd15dbede12ecfc1a3074bd9f2719","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.025Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-96eda23b888c44fb57ff2a93: done: 0.052217 sec","job_status":"done","queue_duration_s":0.974313,"scheduling_latency_s":0.974356,"redis_calls":11,"redis_duration_s":0.004111,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000633,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001923,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001555,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0009556,"exclusive_lock_hold_duration_s":0.01333084,"duration_s":0.052217,"completed_at":"2026-09-06T16:38:51.051Z","load_balancing_strategy":"primary","db_duration_s":0.006192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2141gitlab # [ 483.194941] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.054Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"603b48b33d4a45371036b8ac","created_at":"2026-09-06T16:38:50.092Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d916e48199c4927a2c1c34237f32333d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.116Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-603b48b33d4a45371036b8ac: start","job_status":"start","queue_duration_s":0.938626,"scheduling_latency_s":0.938666}2142gitlab # [ 483.209558] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.088Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"603b48b33d4a45371036b8ac","created_at":"2026-09-06T16:38:50.092Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d916e48199c4927a2c1c34237f32333d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.116Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-603b48b33d4a45371036b8ac: done: 0.03383 sec","job_status":"done","queue_duration_s":0.938626,"scheduling_latency_s":0.938666,"redis_calls":9,"redis_duration_s":0.002857,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000609,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001613,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03383,"completed_at":"2026-09-06T16:38:51.088Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000926,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2143gitlab # [ 483.251778] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.090Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"dd2b16031ca85b0911bcdd90","created_at":"2026-09-06T16:38:50.140Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8aec49c01fcb1232856418f49e45c13","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.155Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-dd2b16031ca85b0911bcdd90: start","job_status":"start","queue_duration_s":0.935302,"scheduling_latency_s":0.935338}2144gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2145gitlab # [ 483.266496] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"dd2b16031ca85b0911bcdd90","created_at":"2026-09-06T16:38:50.140Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e8aec49c01fcb1232856418f49e45c13","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.155Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-dd2b16031ca85b0911bcdd90: done: 0.074632 sec","job_status":"done","queue_duration_s":0.935302,"scheduling_latency_s":0.935338,"redis_calls":12,"redis_duration_s":0.0054009999999999996,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000946,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002668,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001787,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02265448,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.074632,"completed_at":"2026-09-06T16:38:51.165Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005943,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2146gitlab # [ 483.310754] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"1b7b52bd0dfb730218c08fff","created_at":"2026-09-06T16:38:50.160Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"245003663f96c25d0077b01981d877dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.172Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-1b7b52bd0dfb730218c08fff: start","job_status":"start","queue_duration_s":0.996619,"scheduling_latency_s":0.99668}2147gitlab # [ 483.325349] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.202Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2148gitlab # [ 483.328540] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.220Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"1b7b52bd0dfb730218c08fff","created_at":"2026-09-06T16:38:50.160Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"245003663f96c25d0077b01981d877dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.172Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-1b7b52bd0dfb730218c08fff: done: 0.051357 sec","job_status":"done","queue_duration_s":0.996619,"scheduling_latency_s":0.99668,"redis_calls":9,"redis_duration_s":0.004052,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000803,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002368,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051357,"completed_at":"2026-09-06T16:38:51.220Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2149gitlab # [ 483.370755] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.223Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f94f2beceed03707dd701cd9","created_at":"2026-09-06T16:38:50.211Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93dba1be252549c7545d4331eea7d3e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.241Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f94f2beceed03707dd701cd9: start","job_status":"start","queue_duration_s":0.982583,"scheduling_latency_s":0.982627}2150gitlab # [ 483.404228] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.347Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f94f2beceed03707dd701cd9","created_at":"2026-09-06T16:38:50.211Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93dba1be252549c7545d4331eea7d3e1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.241Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f94f2beceed03707dd701cd9: done: 0.123793 sec","job_status":"done","queue_duration_s":0.982583,"scheduling_latency_s":0.982627,"redis_calls":13,"redis_duration_s":0.005261,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001503,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001828,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080133,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0012005,"exclusive_lock_hold_duration_s":0.07495044,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.123793,"completed_at":"2026-09-06T16:38:51.347Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031233,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151gitlab # [ 483.474899] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.349Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8ef5ea26f579dceef3192d9a","created_at":"2026-09-06T16:38:50.269Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"433e6a2a9c528b8a09fa7f124d76995f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.300Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8ef5ea26f579dceef3192d9a: start","job_status":"start","queue_duration_s":1.049621,"scheduling_latency_s":1.049653}2152gitlab # [ 483.491015] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.369Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2153gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.22 seconds)2154gitlab # [ 483.496243] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"8ef5ea26f579dceef3192d9a","created_at":"2026-09-06T16:38:50.269Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"433e6a2a9c528b8a09fa7f124d76995f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.300Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-8ef5ea26f579dceef3192d9a: done: 0.032351 sec","job_status":"done","queue_duration_s":1.049621,"scheduling_latency_s":1.049653,"redis_calls":9,"redis_duration_s":0.003216,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000738,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001761,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032351,"completed_at":"2026-09-06T16:38:51.382Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000883,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2155gitlab # [ 483.540125] gitaly[893]: time="2026-09-06T16:38:51.393Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCGMYQMGW8JR5RKC4BB1J grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:21.291" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:51.391" grpc.time_ms=1.34 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2156gitlab # [ 483.553341] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1402157gitlab # [ 483.556039] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.384Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0eec7097e91d4d72240da6dc","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0eec7097e91d4d72240da6dc: start","job_status":"start","queue_duration_s":1.045358,"scheduling_latency_s":1.045387}2158gitlab # [ 483.568311] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.412Z","message":"No existing merge request to be cleaned up."}2159gitlab # [ 483.569999] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.447Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0eec7097e91d4d72240da6dc","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0eec7097e91d4d72240da6dc: done: 0.063371 sec","job_status":"done","queue_duration_s":1.045358,"scheduling_latency_s":1.045387,"redis_calls":12,"redis_duration_s":0.0047090000000000005,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001472,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002821,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063371,"completed_at":"2026-09-06T16:38:51.447Z","load_balancing_strategy":"primary","db_duration_s":0.010821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 483.609728] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.451Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d7f11fc75f91e17e32f46ecd","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d7f11fc75f91e17e32f46ecd: start","job_status":"start","queue_duration_s":1.112102,"scheduling_latency_s":1.112148}2161gitlab # [ 483.622207] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.478Z","message":"No existing merge request to be cleaned up."}2162gitlab # [ 483.623908] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.497Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d7f11fc75f91e17e32f46ecd","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d7f11fc75f91e17e32f46ecd: done: 0.046633 sec","job_status":"done","queue_duration_s":1.112102,"scheduling_latency_s":1.112148,"redis_calls":10,"redis_duration_s":0.00403,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002431,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032216,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046633,"completed_at":"2026-09-06T16:38:51.497Z","load_balancing_strategy":"primary","db_duration_s":0.00394,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2163gitlab # [ 483.662604] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.500Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"54aede76af42776ed65a125f","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-54aede76af42776ed65a125f: start","job_status":"start","queue_duration_s":1.161287,"scheduling_latency_s":1.161358}2164gitlab # [ 483.674873] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.520Z","message":"No existing merge request to be cleaned up."}2165gitlab # [ 483.676573] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.534Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"54aede76af42776ed65a125f","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-54aede76af42776ed65a125f: done: 0.034496 sec","job_status":"done","queue_duration_s":1.161287,"scheduling_latency_s":1.161358,"redis_calls":10,"redis_duration_s":0.0034349999999999997,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001374,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002061,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034496,"completed_at":"2026-09-06T16:38:51.534Z","load_balancing_strategy":"primary","db_duration_s":0.003387,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166gitlab # [ 483.715784] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.538Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"577d75d3c26796b7bc739748","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-577d75d3c26796b7bc739748: start","job_status":"start","queue_duration_s":1.199161,"scheduling_latency_s":1.199214}2167gitlab # [ 483.728107] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.566Z","message":"No existing merge request to be cleaned up."}2168gitlab # [ 483.729775] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.586Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"577d75d3c26796b7bc739748","created_at":"2026-09-06T16:38:50.254Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-c357dbbe8ab54d94","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3a81acdf33784bcd07fb31c0a981685e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.339Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-577d75d3c26796b7bc739748: done: 0.048414 sec","job_status":"done","queue_duration_s":1.199161,"scheduling_latency_s":1.199214,"redis_calls":10,"redis_duration_s":0.004407,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001926,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002481,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.048414,"completed_at":"2026-09-06T16:38:51.586Z","load_balancing_strategy":"primary","db_duration_s":0.003645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2169gitlab # [ 483.767928] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.589Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"18e988c78d5f40f4ef2644d2","created_at":"2026-09-06T16:38:50.342Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7dbc2d532f17865377172c23a764738","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:38:50.374Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-18e988c78d5f40f4ef2644d2: start","job_status":"start","queue_duration_s":1.21495,"scheduling_latency_s":1.214991}2170gitlab # [ 483.781469] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.613Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2171gitlab # [ 483.784790] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.626Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"18e988c78d5f40f4ef2644d2","created_at":"2026-09-06T16:38:50.342Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f7dbc2d532f17865377172c23a764738","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-06T16:38:50.374Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-18e988c78d5f40f4ef2644d2: done: 0.037151 sec","job_status":"done","queue_duration_s":1.21495,"scheduling_latency_s":1.214991,"redis_calls":9,"redis_duration_s":0.003334,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000885,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037151,"completed_at":"2026-09-06T16:38:51.626Z","load_balancing_strategy":"primary","db_duration_s":0.000841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 483.826430] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.629Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6a4fffc31d0bbef2d7e29374","created_at":"2026-09-06T16:38:50.417Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c8626d0b340c79f6327d68fe1fd05e7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.435Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6a4fffc31d0bbef2d7e29374: start","job_status":"start","queue_duration_s":1.193729,"scheduling_latency_s":1.19376}2173gitlab # [ 483.870442] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6a4fffc31d0bbef2d7e29374","created_at":"2026-09-06T16:38:50.417Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5c8626d0b340c79f6327d68fe1fd05e7","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.435Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6a4fffc31d0bbef2d7e29374: done: 0.035451 sec","job_status":"done","queue_duration_s":1.193729,"scheduling_latency_s":1.19376,"redis_calls":11,"redis_duration_s":0.003418,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000583,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00078648,"exclusive_lock_hold_duration_s":0.00460176,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.035451,"completed_at":"2026-09-06T16:38:51.664Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003006,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2174gitlab # [ 483.945044] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.667Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"436ae974d0e4ee5b3ce79e8c","created_at":"2026-09-06T16:38:50.458Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d208e64a42733a030400729b850eea98","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.482Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-436ae974d0e4ee5b3ce79e8c: start","job_status":"start","queue_duration_s":1.184872,"scheduling_latency_s":1.184902}2175gitlab # [ 483.965200] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"436ae974d0e4ee5b3ce79e8c","created_at":"2026-09-06T16:38:50.458Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d208e64a42733a030400729b850eea98","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.482Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-436ae974d0e4ee5b3ce79e8c: done: 0.039265 sec","job_status":"done","queue_duration_s":1.184872,"scheduling_latency_s":1.184902,"redis_calls":9,"redis_duration_s":0.002548,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001478,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000538,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.039265,"completed_at":"2026-09-06T16:38:51.706Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2176gitlab # [ 484.011748] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:51.708Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"349ca429b028c535507f488e","created_at":"2026-09-06T16:38:50.507Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20d99224afddc0f0499bd51b091538d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.537Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-349ca429b028c535507f488e: start","job_status":"start","queue_duration_s":1.170592,"scheduling_latency_s":1.170621}2177gitlab # [ 484.201883] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"349ca429b028c535507f488e","created_at":"2026-09-06T16:38:50.507Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20d99224afddc0f0499bd51b091538d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.537Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-349ca429b028c535507f488e: done: 0.437243 sec","job_status":"done","queue_duration_s":1.170592,"scheduling_latency_s":1.170621,"redis_calls":564,"redis_duration_s":0.06575800000000001,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00062,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001466,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.063672,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.40438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.437243,"completed_at":"2026-09-06T16:38:52.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2178gitlab # [ 484.297563] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.147Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"398ff3259c0c04b5a88b33ba","created_at":"2026-09-06T16:38:50.562Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d601149a481458c1476a6164538a371f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.584Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-398ff3259c0c04b5a88b33ba: start","job_status":"start","queue_duration_s":1.56276,"scheduling_latency_s":1.562784}2179gitlab # [ 484.315799] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"398ff3259c0c04b5a88b33ba","created_at":"2026-09-06T16:38:50.562Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d601149a481458c1476a6164538a371f","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.584Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-398ff3259c0c04b5a88b33ba: done: 0.033469 sec","job_status":"done","queue_duration_s":1.56276,"scheduling_latency_s":1.562784,"redis_calls":9,"redis_duration_s":0.002447,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000564,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001365,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033469,"completed_at":"2026-09-06T16:38:52.180Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008107,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180gitlab # [ 484.365513] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.182Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"739abeef4276c90e84c68a80","created_at":"2026-09-06T16:38:50.610Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d00443473c64a60169da4c0414c48962","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.626Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-739abeef4276c90e84c68a80: start","job_status":"start","queue_duration_s":1.556303,"scheduling_latency_s":1.556331}2181gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2182gitlab # [ 484.578089] nix-daemon[3985]: accepted connection from pid 5066, user nixbot2183gitlab # [ 484.599193] nix-daemon[3985]: accepted connection from pid 5033, user nixbot2184gitlab # [ 484.613603] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.557Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"739abeef4276c90e84c68a80","created_at":"2026-09-06T16:38:50.610Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d00443473c64a60169da4c0414c48962","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.626Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-739abeef4276c90e84c68a80: done: 0.374112 sec","job_status":"done","queue_duration_s":1.556303,"scheduling_latency_s":1.556331,"redis_calls":13,"redis_duration_s":0.004334,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00135,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001625,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.067,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.067,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.284362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006938,"exclusive_lock_hold_duration_s":0.33777596,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.374112,"completed_at":"2026-09-06T16:38:52.556Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.101033,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2185gitlab # [ 484.701368] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bb2ae4c11b2a06c9f570cb37","created_at":"2026-09-06T16:38:50.676Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8918c960e86f236cdf076c3dd7f7daa0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.706Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bb2ae4c11b2a06c9f570cb37: start","job_status":"start","queue_duration_s":1.853553,"scheduling_latency_s":1.853592}2186gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.25 seconds)2187gitlab # [ 484.718352] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.613Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"bb2ae4c11b2a06c9f570cb37","created_at":"2026-09-06T16:38:50.676Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8918c960e86f236cdf076c3dd7f7daa0","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.706Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-bb2ae4c11b2a06c9f570cb37: done: 0.053713 sec","job_status":"done","queue_duration_s":1.853553,"scheduling_latency_s":1.853592,"redis_calls":9,"redis_duration_s":0.003464,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000865,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001867,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000732,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053713,"completed_at":"2026-09-06T16:38:52.613Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014983,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2188gitlab # [ 484.761941] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.615Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"82735e869cc734fcb984ae83","created_at":"2026-09-06T16:38:50.744Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.776Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-82735e869cc734fcb984ae83: start","job_status":"start","queue_duration_s":1.839795,"scheduling_latency_s":1.839826}2189gitlab # [ 484.776772] gitaly[893]: time="2026-09-06T16:38:52.634Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCHVE69B8MQV2R0TH62XE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:22.531" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:52.632" grpc.time_ms=1.137 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2190gitlab # [ 484.790308] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1582191gitlab # [ 484.810662] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"82735e869cc734fcb984ae83","created_at":"2026-09-06T16:38:50.744Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.776Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-82735e869cc734fcb984ae83: done: 0.138851 sec","job_status":"done","queue_duration_s":1.839795,"scheduling_latency_s":1.839826,"redis_calls":70,"redis_duration_s":0.016427,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.002679,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000953,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.011905,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118977,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.138851,"completed_at":"2026-09-06T16:38:52.754Z","load_balancing_strategy":"primary","db_duration_s":0.003212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2192gitlab # [ 484.858366] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.757Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"af3ba6f27b9073163fcaec99","created_at":"2026-09-06T16:38:50.798Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3866f1ef79b721eeeaf5b89207f97a9a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.822Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-af3ba6f27b9073163fcaec99: start","job_status":"start","queue_duration_s":1.934505,"scheduling_latency_s":1.934536}2193gitlab # [ 484.874054] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"af3ba6f27b9073163fcaec99","created_at":"2026-09-06T16:38:50.798Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3866f1ef79b721eeeaf5b89207f97a9a","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.822Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-af3ba6f27b9073163fcaec99: done: 0.059056 sec","job_status":"done","queue_duration_s":1.934505,"scheduling_latency_s":1.934536,"redis_calls":11,"redis_duration_s":0.0032849999999999997,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000538,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001383,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00086338,"exclusive_lock_hold_duration_s":0.0230695,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.059056,"completed_at":"2026-09-06T16:38:52.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024491,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2194gitlab # [ 484.920521] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.818Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"cb4010f5aa3f6211b6e5ff81","created_at":"2026-09-06T16:38:50.833Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa19bab044599ebc7f9845529b2e6204","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.861Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-cb4010f5aa3f6211b6e5ff81: start","job_status":"start","queue_duration_s":1.95633,"scheduling_latency_s":1.956357}2195gitlab # [ 484.934223] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.840Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"cb4010f5aa3f6211b6e5ff81","created_at":"2026-09-06T16:38:50.833Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-b4f115e83b2c4821","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fa19bab044599ebc7f9845529b2e6204","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:50.861Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-cb4010f5aa3f6211b6e5ff81: done: 0.022538 sec","job_status":"done","queue_duration_s":1.95633,"scheduling_latency_s":1.956357,"redis_calls":11,"redis_duration_s":0.0028,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000535,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013321,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022538,"completed_at":"2026-09-06T16:38:52.840Z","load_balancing_strategy":"primary","db_duration_s":0.000838,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196gitlab # [ 484.979464] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.842Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d3b13b2ec792428bf1ed09d","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d3b13b2ec792428bf1ed09d: start","job_status":"start","queue_duration_s":0.099608,"scheduling_latency_s":0.099636}2197gitlab # [ 484.992732] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d3b13b2ec792428bf1ed09d","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d3b13b2ec792428bf1ed09d: done: 0.026134 sec","job_status":"done","queue_duration_s":0.099608,"scheduling_latency_s":0.099636,"redis_calls":15,"redis_duration_s":0.0038789999999999996,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000933,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002469,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026134,"completed_at":"2026-09-06T16:38:52.868Z","load_balancing_strategy":"primary","db_duration_s":0.000684,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2198gitlab # [ 485.035713] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.870Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19eab1f49e1a0d63f9d3e5d6","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19eab1f49e1a0d63f9d3e5d6: start","job_status":"start","queue_duration_s":0.127388,"scheduling_latency_s":0.127414}2199gitlab # [ 485.048758] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19eab1f49e1a0d63f9d3e5d6","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19eab1f49e1a0d63f9d3e5d6: done: 0.017136 sec","job_status":"done","queue_duration_s":0.127388,"scheduling_latency_s":0.127414,"redis_calls":13,"redis_duration_s":0.002971,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017136,"completed_at":"2026-09-06T16:38:52.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2200gitlab # [ 485.089399] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2cf2905c9022fee569f85f0","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2cf2905c9022fee569f85f0: start","job_status":"start","queue_duration_s":0.14617,"scheduling_latency_s":0.146195}2201gitlab # [ 485.102219] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2cf2905c9022fee569f85f0","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2cf2905c9022fee569f85f0: done: 0.017405 sec","job_status":"done","queue_duration_s":0.14617,"scheduling_latency_s":0.146195,"redis_calls":13,"redis_duration_s":0.0030480000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000999,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017405,"completed_at":"2026-09-06T16:38:52.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 485.142797] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ac62eba2e215830c9df9b4c","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ac62eba2e215830c9df9b4c: start","job_status":"start","queue_duration_s":0.165191,"scheduling_latency_s":0.165217}2203gitlab # [ 485.156998] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ac62eba2e215830c9df9b4c","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ac62eba2e215830c9df9b4c: done: 0.034298 sec","job_status":"done","queue_duration_s":0.165191,"scheduling_latency_s":0.165217,"redis_calls":13,"redis_duration_s":0.005352,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003689,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034298,"completed_at":"2026-09-06T16:38:52.942Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2204gitlab # [ 485.197399] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"090bdbb1bf33da329e82f6cb","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-090bdbb1bf33da329e82f6cb: start","job_status":"start","queue_duration_s":0.202712,"scheduling_latency_s":0.202762}2205gitlab # [ 485.210339] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"090bdbb1bf33da329e82f6cb","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-090bdbb1bf33da329e82f6cb: done: 0.035493 sec","job_status":"done","queue_duration_s":0.202712,"scheduling_latency_s":0.202762,"redis_calls":13,"redis_duration_s":0.003959,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001364,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002595,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035493,"completed_at":"2026-09-06T16:38:52.981Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206gitlab # [ 485.251074] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:52.983Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83bdb8f0c030fd69f9458152","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83bdb8f0c030fd69f9458152: start","job_status":"start","queue_duration_s":0.24038,"scheduling_latency_s":0.240417}2207gitlab # [ 485.264098] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83bdb8f0c030fd69f9458152","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83bdb8f0c030fd69f9458152: done: 0.038664 sec","job_status":"done","queue_duration_s":0.24038,"scheduling_latency_s":0.240417,"redis_calls":13,"redis_duration_s":0.005575999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003619,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038664,"completed_at":"2026-09-06T16:38:53.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208gitlab # [ 485.304703] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3bbc3686e7cdc0c7cf706c8","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3bbc3686e7cdc0c7cf706c8: start","job_status":"start","queue_duration_s":0.282229,"scheduling_latency_s":0.28228}2209gitlab # [ 485.317529] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.064Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3bbc3686e7cdc0c7cf706c8","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3bbc3686e7cdc0c7cf706c8: done: 0.038923 sec","job_status":"done","queue_duration_s":0.282229,"scheduling_latency_s":0.28228,"redis_calls":13,"redis_duration_s":0.006881,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001485,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005396,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038923,"completed_at":"2026-09-06T16:38:53.064Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2210gitlab # [ 485.359049] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.066Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7458c48a0b373d4a04f222","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7458c48a0b373d4a04f222: start","job_status":"start","queue_duration_s":0.323736,"scheduling_latency_s":0.323779}2211gitlab # [ 485.372105] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e7458c48a0b373d4a04f222","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e7458c48a0b373d4a04f222: done: 0.028734 sec","job_status":"done","queue_duration_s":0.323736,"scheduling_latency_s":0.323779,"redis_calls":13,"redis_duration_s":0.003633,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002416,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028734,"completed_at":"2026-09-06T16:38:53.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab # [ 485.412591] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"418f91ae61070816018e5eb2","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-418f91ae61070816018e5eb2: start","job_status":"start","queue_duration_s":0.354611,"scheduling_latency_s":0.354644}2213gitlab # [ 485.425419] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"418f91ae61070816018e5eb2","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-418f91ae61070816018e5eb2: done: 0.022795 sec","job_status":"done","queue_duration_s":0.354611,"scheduling_latency_s":0.354644,"redis_calls":13,"redis_duration_s":0.00327,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002165,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022795,"completed_at":"2026-09-06T16:38:53.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214gitlab # [ 485.465857] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.122Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e934eff08b632c2bed683d7","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e934eff08b632c2bed683d7: start","job_status":"start","queue_duration_s":0.379202,"scheduling_latency_s":0.379231}2215gitlab # [ 485.478371] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e934eff08b632c2bed683d7","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e934eff08b632c2bed683d7: done: 0.01976 sec","job_status":"done","queue_duration_s":0.379202,"scheduling_latency_s":0.379231,"redis_calls":13,"redis_duration_s":0.003019,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011795,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01976,"completed_at":"2026-09-06T16:38:53.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 485.517824] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc124a11057cd66ca01e6e24","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc124a11057cd66ca01e6e24: start","job_status":"start","queue_duration_s":0.400646,"scheduling_latency_s":0.400674}2217gitlab # [ 485.530304] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc124a11057cd66ca01e6e24","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc124a11057cd66ca01e6e24: done: 0.017716 sec","job_status":"done","queue_duration_s":0.400646,"scheduling_latency_s":0.400674,"redis_calls":13,"redis_duration_s":0.0027679999999999996,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001811,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017716,"completed_at":"2026-09-06T16:38:53.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2218gitlab # [ 485.570360] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8005f1c37c0d22c77d57f52","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8005f1c37c0d22c77d57f52: start","job_status":"start","queue_duration_s":0.42028,"scheduling_latency_s":0.420309}2219gitlab # [ 485.582794] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.181Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8005f1c37c0d22c77d57f52","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8005f1c37c0d22c77d57f52: done: 0.017816 sec","job_status":"done","queue_duration_s":0.42028,"scheduling_latency_s":0.420309,"redis_calls":13,"redis_duration_s":0.002874,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000984,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00189,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017816,"completed_at":"2026-09-06T16:38:53.181Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220gitlab # [ 485.621768] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9b6e1bd5006cda815d830c5","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9b6e1bd5006cda815d830c5: start","job_status":"start","queue_duration_s":0.439753,"scheduling_latency_s":0.439781}2221gitlab # [ 485.634216] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9b6e1bd5006cda815d830c5","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9b6e1bd5006cda815d830c5: done: 0.017434 sec","job_status":"done","queue_duration_s":0.439753,"scheduling_latency_s":0.439781,"redis_calls":13,"redis_duration_s":0.002787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001778,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017434,"completed_at":"2026-09-06T16:38:53.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2222gitlab # [ 485.673032] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c91396fcffd2b291908e9542","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c91396fcffd2b291908e9542: start","job_status":"start","queue_duration_s":0.459123,"scheduling_latency_s":0.459152}2223gitlab # [ 485.685381] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c91396fcffd2b291908e9542","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c91396fcffd2b291908e9542: done: 0.017827 sec","job_status":"done","queue_duration_s":0.459123,"scheduling_latency_s":0.459152,"redis_calls":13,"redis_duration_s":0.0029319999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017827,"completed_at":"2026-09-06T16:38:53.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 485.724745] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b5ac71a2a09ebd1284bd549","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b5ac71a2a09ebd1284bd549: start","job_status":"start","queue_duration_s":0.478551,"scheduling_latency_s":0.478579}2225gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2226gitlab # [ 485.737033] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b5ac71a2a09ebd1284bd549","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b5ac71a2a09ebd1284bd549: done: 0.024943 sec","job_status":"done","queue_duration_s":0.478551,"scheduling_latency_s":0.478579,"redis_calls":13,"redis_duration_s":0.008570999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.007199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024943,"completed_at":"2026-09-06T16:38:53.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2227gitlab # [ 485.774133] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d12ecfc1581fbf9503e39226","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d12ecfc1581fbf9503e39226: start","job_status":"start","queue_duration_s":0.50557,"scheduling_latency_s":0.505602}2228gitlab # [ 485.786634] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d12ecfc1581fbf9503e39226","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d12ecfc1581fbf9503e39226: done: 0.020886 sec","job_status":"done","queue_duration_s":0.50557,"scheduling_latency_s":0.505602,"redis_calls":13,"redis_duration_s":0.00363,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002574,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020886,"completed_at":"2026-09-06T16:38:53.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229gitlab # [ 485.825559] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4797918721f0f6b81927ad94","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4797918721f0f6b81927ad94: start","job_status":"start","queue_duration_s":0.528421,"scheduling_latency_s":0.528452}2230gitlab # [ 485.838020] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4797918721f0f6b81927ad94","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4797918721f0f6b81927ad94: done: 0.019911 sec","job_status":"done","queue_duration_s":0.528421,"scheduling_latency_s":0.528452,"redis_calls":13,"redis_duration_s":0.003366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001198,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019911,"completed_at":"2026-09-06T16:38:53.291Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab # [ 485.878025] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a89cec8aebdb966685c0a25a","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a89cec8aebdb966685c0a25a: start","job_status":"start","queue_duration_s":0.550075,"scheduling_latency_s":0.550104}2232gitlab # [ 485.890617] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a89cec8aebdb966685c0a25a","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a89cec8aebdb966685c0a25a: done: 0.018236 sec","job_status":"done","queue_duration_s":0.550075,"scheduling_latency_s":0.550104,"redis_calls":13,"redis_duration_s":0.002872,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001021,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001851,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018236,"completed_at":"2026-09-06T16:38:53.311Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233gitlab # [ 485.930960] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4928fa6c6214e7896f3d29dc","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4928fa6c6214e7896f3d29dc: start","job_status":"start","queue_duration_s":0.570045,"scheduling_latency_s":0.570075}2234gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.24 seconds)2235gitlab # [ 485.943984] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.331Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4928fa6c6214e7896f3d29dc","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4928fa6c6214e7896f3d29dc: done: 0.017927 sec","job_status":"done","queue_duration_s":0.570045,"scheduling_latency_s":0.570075,"redis_calls":13,"redis_duration_s":0.0028770000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000988,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010923,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017927,"completed_at":"2026-09-06T16:38:53.330Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236gitlab # [ 485.985877] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4908e78a864d06f6c6537fa4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4908e78a864d06f6c6537fa4: start","job_status":"start","queue_duration_s":0.589643,"scheduling_latency_s":0.58967}2237gitlab # [ 485.999081] gitaly[893]: time="2026-09-06T16:38:53.876Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCK33K8SGZ202QWD6DC18 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:23.774" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:53.875" grpc.time_ms=1.078 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2238gitlab # [ 486.012789] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1232239gitlab # [ 486.015545] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4908e78a864d06f6c6537fa4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4908e78a864d06f6c6537fa4: done: 0.017767 sec","job_status":"done","queue_duration_s":0.589643,"scheduling_latency_s":0.58967,"redis_calls":13,"redis_duration_s":0.002909,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000979,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00193,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017767,"completed_at":"2026-09-06T16:38:53.350Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2240gitlab # [ 486.056101] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d56ed29118676358c3bc9ea4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d56ed29118676358c3bc9ea4: start","job_status":"start","queue_duration_s":0.60906,"scheduling_latency_s":0.609087}2241gitlab # [ 486.068852] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d56ed29118676358c3bc9ea4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d56ed29118676358c3bc9ea4: done: 0.01854 sec","job_status":"done","queue_duration_s":0.60906,"scheduling_latency_s":0.609087,"redis_calls":13,"redis_duration_s":0.003369,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002396,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010961,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01854,"completed_at":"2026-09-06T16:38:53.370Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2242gitlab # [ 486.109750] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e56eb031c5898d20d8a918","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e56eb031c5898d20d8a918: start","job_status":"start","queue_duration_s":0.629355,"scheduling_latency_s":0.629383}2243gitlab # [ 486.122745] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98e56eb031c5898d20d8a918","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98e56eb031c5898d20d8a918: done: 0.017841 sec","job_status":"done","queue_duration_s":0.629355,"scheduling_latency_s":0.629383,"redis_calls":13,"redis_duration_s":0.003142,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017841,"completed_at":"2026-09-06T16:38:53.390Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2244gitlab # [ 486.163113] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4a2139bcf89bb798edfdd4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4a2139bcf89bb798edfdd4: start","job_status":"start","queue_duration_s":0.649089,"scheduling_latency_s":0.649118}2245gitlab # [ 486.175562] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.410Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1c4a2139bcf89bb798edfdd4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1c4a2139bcf89bb798edfdd4: done: 0.01805 sec","job_status":"done","queue_duration_s":0.649089,"scheduling_latency_s":0.649118,"redis_calls":13,"redis_duration_s":0.00296,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000982,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01805,"completed_at":"2026-09-06T16:38:53.410Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2246gitlab # [ 486.215897] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.411Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7223d8e74b3cfb681a264141","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7223d8e74b3cfb681a264141: start","job_status":"start","queue_duration_s":0.668794,"scheduling_latency_s":0.668821}2247gitlab # [ 486.228428] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.429Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7223d8e74b3cfb681a264141","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7223d8e74b3cfb681a264141: done: 0.017802 sec","job_status":"done","queue_duration_s":0.668794,"scheduling_latency_s":0.668821,"redis_calls":13,"redis_duration_s":0.0029779999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001084,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017802,"completed_at":"2026-09-06T16:38:53.429Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 486.267311] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.431Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5a5a94098f2c300ec34653e","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5a5a94098f2c300ec34653e: start","job_status":"start","queue_duration_s":0.688208,"scheduling_latency_s":0.688234}2249gitlab # [ 486.279763] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.447Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5a5a94098f2c300ec34653e","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5a5a94098f2c300ec34653e: done: 0.016272 sec","job_status":"done","queue_duration_s":0.688208,"scheduling_latency_s":0.688234,"redis_calls":13,"redis_duration_s":0.002792,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000936,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001856,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016272,"completed_at":"2026-09-06T16:38:53.447Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2250gitlab # [ 486.319035] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e282da05213294d329f35b14","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e282da05213294d329f35b14: start","job_status":"start","queue_duration_s":0.706331,"scheduling_latency_s":0.706357}2251gitlab # [ 486.331488] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e282da05213294d329f35b14","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e282da05213294d329f35b14: done: 0.017032 sec","job_status":"done","queue_duration_s":0.706331,"scheduling_latency_s":0.706357,"redis_calls":13,"redis_duration_s":0.0032749999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002315,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017032,"completed_at":"2026-09-06T16:38:53.466Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2252gitlab # [ 486.370771] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bd984fc51cd5134b2f3e4a6","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bd984fc51cd5134b2f3e4a6: start","job_status":"start","queue_duration_s":0.725,"scheduling_latency_s":0.725028}2253gitlab # [ 486.383160] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bd984fc51cd5134b2f3e4a6","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bd984fc51cd5134b2f3e4a6: done: 0.01646 sec","job_status":"done","queue_duration_s":0.725,"scheduling_latency_s":0.725028,"redis_calls":13,"redis_duration_s":0.002838,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010023,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01646,"completed_at":"2026-09-06T16:38:53.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2254gitlab # [ 486.423319] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62bb3597befc75ac24ded4c9","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62bb3597befc75ac24ded4c9: start","job_status":"start","queue_duration_s":0.743086,"scheduling_latency_s":0.743111}2255gitlab # [ 486.435806] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62bb3597befc75ac24ded4c9","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62bb3597befc75ac24ded4c9: done: 0.016199 sec","job_status":"done","queue_duration_s":0.743086,"scheduling_latency_s":0.743111,"redis_calls":13,"redis_duration_s":0.002673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001728,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016199,"completed_at":"2026-09-06T16:38:53.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256gitlab # [ 486.475079] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a4fac893102eada74ac6e69","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a4fac893102eada74ac6e69: start","job_status":"start","queue_duration_s":0.761099,"scheduling_latency_s":0.761126}2257gitlab # [ 486.487533] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a4fac893102eada74ac6e69","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a4fac893102eada74ac6e69: done: 0.016267 sec","job_status":"done","queue_duration_s":0.761099,"scheduling_latency_s":0.761126,"redis_calls":13,"redis_duration_s":0.002744,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00094,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001804,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016267,"completed_at":"2026-09-06T16:38:53.520Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258gitlab # [ 486.526670] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32fe8a4b58e8bfd5b62ebe35","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32fe8a4b58e8bfd5b62ebe35: start","job_status":"start","queue_duration_s":0.779364,"scheduling_latency_s":0.779394}2259gitlab # [ 486.539117] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.539Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32fe8a4b58e8bfd5b62ebe35","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32fe8a4b58e8bfd5b62ebe35: done: 0.017469 sec","job_status":"done","queue_duration_s":0.779364,"scheduling_latency_s":0.779394,"redis_calls":13,"redis_duration_s":0.0032370000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001355,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010943,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017469,"completed_at":"2026-09-06T16:38:53.539Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2260gitlab # [ 486.578432] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff4a80573e513abc3e3020ae","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff4a80573e513abc3e3020ae: start","job_status":"start","queue_duration_s":0.79844,"scheduling_latency_s":0.798467}2261gitlab # [ 486.590786] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff4a80573e513abc3e3020ae","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff4a80573e513abc3e3020ae: done: 0.027788 sec","job_status":"done","queue_duration_s":0.79844,"scheduling_latency_s":0.798467,"redis_calls":13,"redis_duration_s":0.003677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002523,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027788,"completed_at":"2026-09-06T16:38:53.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262gitlab # [ 486.630966] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e4407d6256c2c8462a96527","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e4407d6256c2c8462a96527: start","job_status":"start","queue_duration_s":0.829484,"scheduling_latency_s":0.829537}2263gitlab # [ 486.643469] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e4407d6256c2c8462a96527","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e4407d6256c2c8462a96527: done: 0.257392 sec","job_status":"done","queue_duration_s":0.829484,"scheduling_latency_s":0.829537,"redis_calls":13,"redis_duration_s":0.211278,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.111953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.099325,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.257392,"completed_at":"2026-09-06T16:38:53.829Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2264gitlab # [ 486.682807] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99dc7bcd38c2d7488ef44ae0","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99dc7bcd38c2d7488ef44ae0: start","job_status":"start","queue_duration_s":1.091268,"scheduling_latency_s":1.091325}2265gitlab # [ 486.695272] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.869Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99dc7bcd38c2d7488ef44ae0","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99dc7bcd38c2d7488ef44ae0: done: 0.034969 sec","job_status":"done","queue_duration_s":1.091268,"scheduling_latency_s":1.091325,"redis_calls":13,"redis_duration_s":0.004918,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001861,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003057,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.034969,"completed_at":"2026-09-06T16:38:53.869Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2266gitlab # [ 486.735018] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcbcb83392e63289b00b8bf4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcbcb83392e63289b00b8bf4: start","job_status":"start","queue_duration_s":1.128808,"scheduling_latency_s":1.128849}2267gitlab # [ 486.747763] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcbcb83392e63289b00b8bf4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcbcb83392e63289b00b8bf4: done: 0.02779 sec","job_status":"done","queue_duration_s":1.128808,"scheduling_latency_s":1.128849,"redis_calls":13,"redis_duration_s":0.003857,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001335,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002522,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02779,"completed_at":"2026-09-06T16:38:53.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2268gitlab # [ 486.787531] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baa4453410214077bf127884","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baa4453410214077bf127884: start","job_status":"start","queue_duration_s":1.15876,"scheduling_latency_s":1.158791}2269gitlab # [ 486.800189] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"baa4453410214077bf127884","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-baa4453410214077bf127884: done: 0.026582 sec","job_status":"done","queue_duration_s":1.15876,"scheduling_latency_s":1.158791,"redis_calls":13,"redis_duration_s":0.006512,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004364,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002148,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026582,"completed_at":"2026-09-06T16:38:53.928Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2270gitlab # [ 486.840412] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61ab88f34c034397c93e8700","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61ab88f34c034397c93e8700: start","job_status":"start","queue_duration_s":1.187659,"scheduling_latency_s":1.187692}2271gitlab # [ 486.852614] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61ab88f34c034397c93e8700","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61ab88f34c034397c93e8700: done: 0.020908 sec","job_status":"done","queue_duration_s":1.187659,"scheduling_latency_s":1.187692,"redis_calls":13,"redis_duration_s":0.003211,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001111,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012888,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020908,"completed_at":"2026-09-06T16:38:53.951Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2272gitlab # [ 486.891964] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5d64db17d0d235d71bec85c","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5d64db17d0d235d71bec85c: start","job_status":"start","queue_duration_s":1.210547,"scheduling_latency_s":1.210577}2273gitlab # [ 486.904499] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5d64db17d0d235d71bec85c","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5d64db17d0d235d71bec85c: done: 0.020372 sec","job_status":"done","queue_duration_s":1.210547,"scheduling_latency_s":1.210577,"redis_calls":13,"redis_duration_s":0.003079,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001276,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001803,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012773,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020372,"completed_at":"2026-09-06T16:38:53.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2274gitlab # [ 486.943554] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e6399cecaa718f7a6795048","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e6399cecaa718f7a6795048: start","job_status":"start","queue_duration_s":1.232747,"scheduling_latency_s":1.232777}2275gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2276gitlab # [ 486.956113] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7e6399cecaa718f7a6795048","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7e6399cecaa718f7a6795048: done: 0.019554 sec","job_status":"done","queue_duration_s":1.232747,"scheduling_latency_s":1.232777,"redis_calls":13,"redis_duration_s":0.0031390000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00106,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002079,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019554,"completed_at":"2026-09-06T16:38:53.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2277gitlab # [ 486.994490] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:53.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc77ee676a37d3f1744329a4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc77ee676a37d3f1744329a4: start","job_status":"start","queue_duration_s":1.254146,"scheduling_latency_s":1.254175}2278gitlab # [ 487.005963] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc77ee676a37d3f1744329a4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc77ee676a37d3f1744329a4: done: 0.120595 sec","job_status":"done","queue_duration_s":1.254146,"scheduling_latency_s":1.254175,"redis_calls":13,"redis_duration_s":0.003397,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002211,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.120595,"completed_at":"2026-09-06T16:38:54.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 487.044166] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9145e9a47bc970c240a2bfa4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9145e9a47bc970c240a2bfa4: start","job_status":"start","queue_duration_s":1.376626,"scheduling_latency_s":1.376656}2280gitlab # [ 487.056358] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9145e9a47bc970c240a2bfa4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9145e9a47bc970c240a2bfa4: done: 0.021799 sec","job_status":"done","queue_duration_s":1.376626,"scheduling_latency_s":1.376656,"redis_calls":13,"redis_duration_s":0.003306,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002243,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021799,"completed_at":"2026-09-06T16:38:54.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2281gitlab # [ 487.095139] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.143Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9866131ab26c5775553e5590","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9866131ab26c5775553e5590: start","job_status":"start","queue_duration_s":1.400351,"scheduling_latency_s":1.400383}2282gitlab # [ 487.107532] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9866131ab26c5775553e5590","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9866131ab26c5775553e5590: done: 0.021672 sec","job_status":"done","queue_duration_s":1.400351,"scheduling_latency_s":1.400383,"redis_calls":13,"redis_duration_s":0.004134,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021672,"completed_at":"2026-09-06T16:38:54.164Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2283gitlab # [ 487.146512] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624c1089e4c893cc8018719f","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624c1089e4c893cc8018719f: start","job_status":"start","queue_duration_s":1.423913,"scheduling_latency_s":1.423941}2284gitlab # [ 487.158945] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624c1089e4c893cc8018719f","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624c1089e4c893cc8018719f: done: 0.022689 sec","job_status":"done","queue_duration_s":1.423913,"scheduling_latency_s":1.423941,"redis_calls":13,"redis_duration_s":0.003102,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012622,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022689,"completed_at":"2026-09-06T16:38:54.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2285gitlab # [ 487.198035] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d455f3b865291d854a4f7b19","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d455f3b865291d854a4f7b19: start","job_status":"start","queue_duration_s":1.448536,"scheduling_latency_s":1.448564}2286gitlab # [ 487.210419] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.213Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d455f3b865291d854a4f7b19","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d455f3b865291d854a4f7b19: done: 0.021723 sec","job_status":"done","queue_duration_s":1.448536,"scheduling_latency_s":1.448564,"redis_calls":13,"redis_duration_s":0.003263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001117,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013914,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021723,"completed_at":"2026-09-06T16:38:54.213Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2287gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.28 seconds)2288gitlab # [ 487.250650] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"070d45ab5879b0ae33bcbb68","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-070d45ab5879b0ae33bcbb68: start","job_status":"start","queue_duration_s":1.472083,"scheduling_latency_s":1.472111}2289gitlab # [ 487.263197] gitaly[893]: time="2026-09-06T16:38:55.140Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCM9GN59AKJJT6HGRNRKJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:25.038" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:55.138" grpc.time_ms=1.439 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2290gitlab # [ 487.275999] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:55 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1702291gitlab # [ 487.278644] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"070d45ab5879b0ae33bcbb68","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-070d45ab5879b0ae33bcbb68: done: 0.022903 sec","job_status":"done","queue_duration_s":1.472083,"scheduling_latency_s":1.472111,"redis_calls":13,"redis_duration_s":0.003223,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002159,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014118,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022903,"completed_at":"2026-09-06T16:38:54.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2292gitlab # [ 487.318589] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da33165599313ef54e012e94","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da33165599313ef54e012e94: start","job_status":"start","queue_duration_s":1.49693,"scheduling_latency_s":1.496959}2293gitlab # [ 487.331202] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"da33165599313ef54e012e94","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-da33165599313ef54e012e94: done: 0.021902 sec","job_status":"done","queue_duration_s":1.49693,"scheduling_latency_s":1.496959,"redis_calls":13,"redis_duration_s":0.003219,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021902,"completed_at":"2026-09-06T16:38:54.261Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2294gitlab # [ 487.370755] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3440e8fdade8875eb4b0a423","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3440e8fdade8875eb4b0a423: start","job_status":"start","queue_duration_s":1.520895,"scheduling_latency_s":1.520927}2295gitlab # [ 487.383365] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3440e8fdade8875eb4b0a423","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3440e8fdade8875eb4b0a423: done: 0.021856 sec","job_status":"done","queue_duration_s":1.520895,"scheduling_latency_s":1.520927,"redis_calls":13,"redis_duration_s":0.003175,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001102,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002073,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021856,"completed_at":"2026-09-06T16:38:54.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2296gitlab # [ 487.423055] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"283b02ac371728784558e455","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-283b02ac371728784558e455: start","job_status":"start","queue_duration_s":1.545782,"scheduling_latency_s":1.545814}2297gitlab # [ 487.435677] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"283b02ac371728784558e455","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-283b02ac371728784558e455: done: 0.022174 sec","job_status":"done","queue_duration_s":1.545782,"scheduling_latency_s":1.545814,"redis_calls":13,"redis_duration_s":0.0032589999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001179,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022174,"completed_at":"2026-09-06T16:38:54.310Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2298gitlab # [ 487.476614] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22ccdca9e9bb8348b688014b","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22ccdca9e9bb8348b688014b: start","job_status":"start","queue_duration_s":1.569868,"scheduling_latency_s":1.569899}2299gitlab # [ 487.489017] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22ccdca9e9bb8348b688014b","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22ccdca9e9bb8348b688014b: done: 0.02156 sec","job_status":"done","queue_duration_s":1.569868,"scheduling_latency_s":1.569899,"redis_calls":13,"redis_duration_s":0.003085,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00109,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001995,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013835,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02156,"completed_at":"2026-09-06T16:38:54.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2300gitlab # [ 487.527757] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b19f05dc78be0645b43c45e2","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b19f05dc78be0645b43c45e2: start","job_status":"start","queue_duration_s":1.593313,"scheduling_latency_s":1.593341}2301gitlab # [ 487.540234] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b19f05dc78be0645b43c45e2","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b19f05dc78be0645b43c45e2: done: 0.022554 sec","job_status":"done","queue_duration_s":1.593313,"scheduling_latency_s":1.593341,"redis_calls":13,"redis_duration_s":0.002901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001815,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022554,"completed_at":"2026-09-06T16:38:54.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2302gitlab # [ 487.579221] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.360Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1b019d5aa8ae8a82d7287c4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1b019d5aa8ae8a82d7287c4: start","job_status":"start","queue_duration_s":1.617772,"scheduling_latency_s":1.617801}2303gitlab # [ 487.591721] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:54.383Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1b019d5aa8ae8a82d7287c4","created_at":"2026-09-06T16:38:52.645Z","trace_propagation_headers":{"sentry-trace":"a3d22dd076e74a2ca91d92eb34c95b30-d96546e5a78948f7","baggage":"sentry-trace_id=a3d22dd076e74a2ca91d92eb34c95b30,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ac87c3c462dfdfd4e6d3a036330773d6","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-06T16:38:52.743Z","job_size_bytes":2,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1b019d5aa8ae8a82d7287c4: done: 0.022568 sec","job_status":"done","queue_duration_s":1.617772,"scheduling_latency_s":1.617801,"redis_calls":13,"redis_duration_s":0.003222,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002072,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022568,"completed_at":"2026-09-06T16:38:54.383Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2304gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2305gitlab # [ 488.450837] gitaly[893]: time="2026-09-06T16:38:56.393Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCNGNM13N6HJ5D00Y59NP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:26.391" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:56.392" grpc.time_ms=1.353 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2306gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.24 seconds)2307gitlab # [ 488.507146] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1722308gitlab # [ 489.132773] nix-daemon[3985]: accepted connection from pid 5033, user nixbot2309gitlab # [ 489.347709] gitaly[893]: time="2026-09-06T16:38:57.291Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCPCAVEDBCSN3C1QAT5QN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:27.289" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:57.290" grpc.time_ms=1.103 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2310gitlab # [ 489.474420] gitaly[893]: time="2026-09-06T16:38:57.417Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCPCAVEDBCSN3C1QAT5QN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:39:07.009" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:57.400" grpc.time_ms=16.884 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2311gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2312gitlab # [ 489.568819] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:57 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 3472313gitlab # [ 489.573772] nixbot[3813]: {"ts": "2026-09-06T16:38:57.514Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace \"HTTP/1.1 201 Created\""}2314gitlab # [ 489.624617] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:57.567Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"412a0733c2d0be5a2e124dd2","created_at":"2026-09-06T16:38:57.481Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-af0bc105cddf4e15","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:57.487Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-412a0733c2d0be5a2e124dd2: start","job_status":"start","queue_duration_s":0.079371,"scheduling_latency_s":0.079436}2315gitlab # [ 489.657193] gitaly[893]: time="2026-09-06T16:38:57.601Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCPPVHQ52BT5044572ZMY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:27.499" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:57.600" grpc.time_ms=0.892 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2316gitlab # [ 489.683105] gitaly[893]: time="2026-09-06T16:38:57.627Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCPQMQPJ2NAYZKGJFVVGZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:27.625" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:57.626" grpc.time_ms=0.974 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2317gitlab # [ 489.704364] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 1452318gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.20 seconds)2319gitlab # [ 489.758320] gitaly[893]: time="2026-09-06T16:38:57.702Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCPQMQPJ2NAYZKGJFVVGZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:39:07.128" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:57.688" grpc.time_ms=13.586 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2320gitlab # [ 489.836681] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:57 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 2552321gitlab # [ 489.842872] nixbot[3813]: {"ts": "2026-09-06T16:38:57.782Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace \"HTTP/1.1 201 Created\""}2322gitlab # [ 489.948161] gitaly[893]: time="2026-09-06T16:38:57.892Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCQ03Q0HVTPVY73NP2CP2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:27.790" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:57.891" grpc.time_ms=1.042 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2323gitlab # [ 490.037590] gitaly[893]: time="2026-09-06T16:38:57.981Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCQ03Q0HVTPVY73NP2CP2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-06T16:39:07.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-06T16:38:57.965" grpc.time_ms=16.535 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2324gitlab # [ 490.163766] gitaly[893]: time="2026-09-06T16:38:58.107Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCPCAVEDBCSN3C1QAT5QN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-06T16:39:28.105" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:58.106" grpc.time_ms=1.776 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2325gitlab # [ 490.197380] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 3292326gitlab # [ 490.202546] nixbot[3813]: {"ts": "2026-09-06T16:38:58.127Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4061fd525383d96028769d7d55bafd8ab2bd0ace \"HTTP/1.1 201 Created\""}2327gitlab # [ 490.308429] gitaly[893]: time="2026-09-06T16:38:58.252Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCPCAVEDBCSN3C1QAT5QN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:28.250" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:58.251" grpc.time_ms=1.152 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2328gitlab # [ 490.374666] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.318Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"b315db0ce245a37adc8675a6","created_at":"2026-09-06T16:38:58.303Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-e0c7c804dc0e4b2c","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"412a0733c2d0be5a2e124dd2","job_size_bytes":3,"pid":3677,"job_status":"deduplicated","message":"PipelineProcessWorker JID-b315db0ce245a37adc8675a6: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2329gitlab # [ 490.416651] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.332Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"412a0733c2d0be5a2e124dd2","created_at":"2026-09-06T16:38:57.481Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-af0bc105cddf4e15","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:57.487Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3677,"job_status":"rescheduled","message":"PipelineProcessWorker JID-412a0733c2d0be5a2e124dd2: rescheduled"}2330gitlab # [ 490.461951] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.405Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"412a0733c2d0be5a2e124dd2","created_at":"2026-09-06T16:38:57.481Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-af0bc105cddf4e15","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:57.487Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-412a0733c2d0be5a2e124dd2: done: 0.838686 sec","job_status":"done","queue_duration_s":0.079371,"scheduling_latency_s":0.079436,"gitaly_calls":2,"gitaly_duration_s":0.057582,"redis_calls":43,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.097558,"redis_read_bytes":167,"redis_write_bytes":8132,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00999,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.018994,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":6,"redis_queues_duration_s":0.012412,"redis_queues_read_bytes":6,"redis_queues_write_bytes":3275,"redis_queues_metadata_calls":12,"redis_queues_metadata_duration_s":0.034479,"redis_queues_metadata_read_bytes":91,"redis_queues_metadata_write_bytes":1640,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.0014,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.004006,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006287,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":29,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":29,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.337,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":15,"db_ci_count":14,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.093,"db_ci_duration_s":0.244,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.301797,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.44499168,"duration_s":0.838686,"completed_at":"2026-09-06T16:38:58.405Z","load_balancing_strategy":"primary","db_duration_s":0.390989,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2331gitlab # [ 490.566157] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.418Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"37d2c282fbc6d499b72f07ce","created_at":"2026-09-06T16:38:58.023Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-e0c7c804dc0e4b2c","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:58.034Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-37d2c282fbc6d499b72f07ce: start","job_status":"start","queue_duration_s":0.383229,"scheduling_latency_s":0.383268}2332gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses2333gitlab # [ 490.673881] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.615Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"37d2c282fbc6d499b72f07ce","created_at":"2026-09-06T16:38:58.023Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-e0c7c804dc0e4b2c","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:58.034Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-37d2c282fbc6d499b72f07ce: done: 0.197485 sec","job_status":"done","queue_duration_s":0.383229,"scheduling_latency_s":0.383268,"redis_calls":7,"redis_duration_s":0.009554,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004651,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004903,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.197485,"completed_at":"2026-09-06T16:38:58.615Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.044035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2334gitlab # [ 490.748343] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.630Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"e197bfd1a6a2591535596370","created_at":"2026-09-06T16:38:58.061Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-e0c7c804dc0e4b2c","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:58.069Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e197bfd1a6a2591535596370: start","job_status":"start","queue_duration_s":0.559881,"scheduling_latency_s":0.559948}2335gitlab # [ 490.769701] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 762336gitlab # [ 490.772706] gitaly[893]: time="2026-09-06T16:38:58.643Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCQR8S0YV0NRCMEQ02ZW8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:58.633" grpc.time_ms=9.942 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2337gitlab # [ 490.791283] gitaly[893]: time="2026-09-06T16:38:58.735Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCQTPBCNJERDFF3T7Q1EJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:58.718" pid=893 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2338gitlab # [ 490.802780] gitaly[893]: time="2026-09-06T16:38:58.735Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCQTPBCNJERDFF3T7Q1EJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:58.718" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2339gitlab # [ 490.817686] gitlab-workhorse[823]: time="2026-09-06T16:38:58Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672340gitlab # [ 490.819809] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:58 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 902341gitlab # [ 490.824775] gitaly[893]: time="2026-09-06T16:38:58.737Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=5 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCQTPBCNJERDFF3T7Q1EJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:58.718" grpc.time_ms=18.124 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2342gitlab # [ 490.839936] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.775Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2"],"class":"PipelineMetricsWorker","jid":"e197bfd1a6a2591535596370","created_at":"2026-09-06T16:38:58.061Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-e0c7c804dc0e4b2c","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:0017b95f5c428943017bb18cdb04f3707414ec48b7350aa2a2e06a6849a9ea01","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:58.069Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-e197bfd1a6a2591535596370: done: 0.145692 sec","job_status":"done","queue_duration_s":0.559881,"scheduling_latency_s":0.559948,"redis_calls":7,"redis_duration_s":0.005784,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004745,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.022,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.145692,"completed_at":"2026-09-06T16:38:58.775Z","load_balancing_strategy":"primary","db_duration_s":0.030082,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2343gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses, in 0.18 seconds)2344(finished: subtest: eval and build post commit statuses, in 11.30 seconds)2345(finished: run the VM test script, in 564.00 seconds)2346gitlab # [ 490.882790] sidekiq-cluster[3677]: {"severity":"INFO","time":"2026-09-06T16:38:58.778Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["2"],"class":"PipelineProcessWorker","jid":"af2258ccc40fbc4162ccc2df","created_at":"2026-09-06T16:38:58.334Z","trace_propagation_headers":{"sentry-trace":"74656fefedca47d49931a171bca2c7c5-e0c7c804dc0e4b2c","baggage":"sentry-trace_id=74656fefedca47d49931a171bca2c7c5,sentry-release=Unknown"},"correlation_id":"01M1VSCPCAVEDBCSN3C1QAT5QN","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-06T16:38:58.349Z","job_size_bytes":3,"pid":3677,"sidekiq_tid":"m8tx","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-af2258ccc40fbc4162ccc2df: start","job_status":"start","queue_duration_s":0.428288,"scheduling_latency_s":0.428329}2347gitlab # [ 490.898668] gitaly[893]: time="2026-09-06T16:38:58.790Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M1VSCQWPBW39AHDFGV8PVC5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-06T16:39:28.789" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-06T16:38:58.790" grpc.time_ms=0.592 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2348gitlab # [ 490.911929] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4061fd525383d96028769d7d55bafd8ab2bd0ace/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1152349gitlab # [ 490.933911] gitaly[893]: time="2026-09-06T16:38:58.878Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M1VSCQZC0YHR6RC1CFWD4STK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-06T16:38:58.868" grpc.time_ms=9.164 pid=893 remote_ip="2001:db8:1::1" span.kind=server system=grpc2350gitlab # [ 490.948686] gitlab-workhorse[823]: gitlab 2001:db8:1::1 - - [2026/09/06:16:38:58 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 812351test script finished in 564.07s2352cleanup2353kill QemuMachine (pid 46)2354gitlab # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2355(finished: cleanup, in 1.81 seconds)2356additionally exposed symbols:2357 gitlab,2358 vlan1,2359 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2360[]2361[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-06T16:37:15.166Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2362[]2363[]2364[]2365[]2366[]2367[]2368[]2369[]2370[]2371[]2372[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-06T16:38:47.291804Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2373[]2374[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2375[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2376[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2377[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2378[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2379[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2380[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2381[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': '2026-09-06T16:38:57.464Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2382[{'id': 5, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-06T16:38:47.805Z', 'started_at': None, 'finished_at': '2026-09-06T16:38:57.464Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '4061fd525383d96028769d7d55bafd8ab2bd0ace', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-06T16:38:57.741Z', 'started_at': None, 'finished_at': '2026-09-06T16:38:58.054Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]